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Aker, G., Jahns, I., & Schroeder, W. (1968). Evaluation of an adult basic 
education program in a southern rural community. Tallahassee, FL: Florida 
State University, Department of Adult Education. (ERIC Document 
Reproduction Service No. ED 388 780) 

Purpose: This evaluation was the second phase of an evaluative study of an adult 
basic education program for seasonal farm workers. Phase I used observations and 
a case-study approach to make recommendations about student screening, 
curriculum, instructional staff, and program expansion. 

Variables: Outcome variables were grade-level progression and student 
satisfaction. Quality of learning was measured by a Likert scale on a 
questionnaire. Sources of greatest satisfaction were measured by indicating 
“specific subject matter,” “learning in general,” “prepare for a job,” “relate to 
other people,” “miscellaneous,” and “no response.” Other variables included the 
rate of student dropout, appraisals of instructional staff, and the relationship 
between program success and the characteristics of students and instructional 
staff. 

Population: A random sample yielded 305 students (N= 305). Fifty-three percent 
were male. Most of the students were between 40 and 59 years old (56%). The 
mean pretest grade-level distribution was 2.6. A 33% representative sampling 
procedure assured a relative distribution of students in rural and urban centers. 

Design: This study was a pre-post comparison. 

Data Collection: The data were collected via standardized tests, questionnaires, 
and interview schedules. Existing student records were also used. 

Instrumentation: The standardized test used to measure reading gains was not 
mentioned. Quality of learning was measured by a Likert scale. 

Findings: All students who persisted in the program made some type of reading 
gain. The distribution range at pretest was 0.0-8. 1 . The posttest range was 1.1- 
1 1.4. The mean increased from 2.6 to 4.9; the median from 2.7 to 4.5. A majority 
gained from 1 .0 to 3.9 grade levels during the course of the program. Students 
who received the lower pretest scores were more likely to achieve at the higher 
levels. A t test by program site revealed that each center secured a mean gain 
which could have been due to chance less than one time in a hundred. 
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Fifty-six percent of the students indicated that their learning resulted from 
participation in the program and that “very much” learning had taken place. The 
retention rate of students was 72.4%. Age was negatively associate with dropout. 
The older the student, the less the likelihood of dropping out. 

Strengths: The research design was competent. Researchers examined the 
relationship between characteristics of students and staff and program success. 

Weaknesses: The standardized reading test was not identified. Questionnaires 
seemed poorly designed and their methodology was not described. Program 
success was measured by only two criteria, grade-level gain and rate of dropout. 
There was no control group. 

Alamprese, J. (1987). Investing in change: Competency-based adult education 
in California. Sacramento, CA: California State Department of Education. 

Purpose: This study had two purposes: (a) to determine how 306 agency 
personnel had interpreted California’s competency-based adult education (CBAE) 
mandate, and (b) to ascertain the effects of the mandate on agency management, 
classroom functioning, and student outcomes. Only the portion dealing with 
student outcomes is abstracted here. 

Variables: The variables were tested learning gain and retention. 

Population: The population was adult literacy and ESL students in California. For 
tested learning gain, the population was 3,950 in 1983-84 and 6,413 in 1984-85. 

Design: The design was pre- and posttesting for learning gain. Program and state 
records were used for retention. 

Data Collection: Students were posttested after 1 00 hours of instruction. A 
sample of programs identified as high-implementing on the CBAE mandate was 
compared to a sample of low-implementing programs. 

Instrumentation: The CASAS achievement test was used. 

Findings: In 1983-84, the average gain on the CASAS was 6.8. In 1984-85, the 
gain for high-implementing agencies was 5.0; it was 3.5 for low-implementing 
agencies. Retention rates in 1983-84 were 50% (N= 446 classes), 59% in 1983-85 
(N = 561 classes), and 49% in 1984-85 (N= 984 classes). At 100 hours, the 
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retention rate for high-implementing programs was 66.5% versus 47.4% for low- 
implementing programs. 

Strengths: The sample was a respectable size. 

Weaknesses: There were few outcome variables and no control group. 

Alamprese, J. (1993). Systematizing adult education: Final report of the 
Connecticut Adult Performance Program (CAPP). Washington, DC: 
COSMOS Corp. 

Purpose: This document reported the formative evaluation of the Connecticut 
Adult Performance Program (CAPP), a five-year initiative to improve the state’s 
adult literacy education. CAPP was a comprehensive system that included 
program management, staff development, etc. The evaluation was designed to 
collect information on the areas of program operation that required technical 
assistance and training; assess the impact of CAPP on program management and 
operations; and determine the effectiveness of the Bureau of Adult Education and 
Training in conducting the CAPP initiative. Although much of the evaluation and 
report focused on management issues, a portion focused on learner outcomes. 
That portion is reported here. 

Variables: Measurements included tested growth in basic skills and learners’ 
attainment of their individual goals. 

Population: The sample consisted of adult literacy education students enrolled in 
a life-skills program adapted from CASAS. Students were pre- and posttested 
from 1987 through 1991. Numbers varied by year and by the level of instruction. 
In 1987-1988, there were 482 valid cases for reading and 197 for listening. In 
1990-91, there were 1,068 valid cases for reading and 1,879 for listening. 
Students’ attainment of personal goals was also measured through a survey. The 
total population was 1,271 for 1987-88; 1,771 for 1988-89; 2,714 for 1989-90; 
and 11,001 for 1990-91. 

Design: The design was a longitudinal pre- and posttesting comparison for 
learning gain. 

Data Collection: A multi-faceted data collection process was designed which 
included development of monitoring instruments, site observations, analysis of 
program and student assessment information and participation in state-sponsored 
meetings. 
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Instrumentation: The test, known as the CAPP Survey Achievement Pre and Post, 
was adapted from the CASAS test. The test measured gains in reading and 
listening. Data on the reliability and validity of the test were not reported. Data on 
learners’ achievement of their personal goals were collected using a survey 
instrument, CAPP Goal Attainment. A yearly survey was used to measure 
attainment of personal learning goals. 

Findings: Levels a and b gain scores were highest for reading in 1987 (9.9 for 
level a; 7.5 for level b ), and declined in 1988 and 1989. In 1990 scores increased 
to 7.2 for level a and 6. 1 for level b. Gain scores for level c ranged from 4.1 in 
1987 to 4.6 in 1990. For listening, gain scores for level a in 1987 were the highest 
(10.1) and declined to the five-point range in subsequent years. Gains for level b 
were also highest in 1987 and declined in future years. Scores for level c were 
highest in 1 988 and declined to about the 1 .5 range thereafter. Although tests of 
statistical significance were not reported, the substantial sample size would 
suggest that the gains were significantly different. 

Data from the survey showed yearly retention figures of 63% for 1 987, 59% for 
1988, 66% for 1989, and 42% for 1991. In 1990, 15% of all students (both 
retained and no longer active) achieved their personal goals and left the program, 
14% left the program because of barriers, and 30% left the program for unknown 
reasons. 

Strengths: The sample size was relatively large. Yearly data were collected over 
four years. 

Weaknesses: Fine points of design and methodology were not reported. There was 
no control or comparison group. The test for learning gain was unique; hence, 
gain can not be compared with data from other programs. The time period 
between pre- and posttesting was not reported. 

Alpena Community College. (1993). Alpena Community College Workplace 
Partnership Project. MI: Author. (ERIC Document Reproduction Service No. 
ED 356 345) Alpena: MI. 

Purpose: This project was funded by the National Workplace Literacy Program 
from May 31, 1991 to January 31, 1993. The partnership included the college, 
Besser Company, Baker Company, and Fletcher Paper Company. The work-site 
classes included reading, mathematics, and communications. Class content was 
based on the needs of adult learners. Several of the mathematics courses were 
rather advanced (e.g., trigonometry and algebra). Classes were held at the work 
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site and consisted of 16 contact hours. At one site the classes were on-the-clock; 
at two sites they were off-the-clock. 

Variables: Reading gains and attitudes regarding the classes (including some 
measures on how the classes impacted the participants) were measured. 

Population: Two hundred and thirty adults participated in the classes. The 
majority (176) worked for the Besser Company. 

Design: The design was pre-post comparison and evaluated by an independent 
consultant. 

Data Collection: Project staff collected the data. However, for a variety of 
reasons, they discontinued administering the tests halfway through the project. 

The pre- and posttest data were analyzed using t tests for related samples. Each 
subject area was analyzed separately. 

Instrumentation: The Educational Testing Service’s Tests of Applied Literacy 
Skills (TALS) were used to measure reading and math gains. Attitudes were 
measured quantitatively by a Likert scale in a questionnaire and qualitatively by 
an instrument with open-ended questions. 

Findings: Analyses of the pre- and posttest data of reading, math, algebra, and 
communication scores showed no significant increases, although the posttest 
means were higher. Learners’ self-esteem increased. All responded positively to 
the statement, “You feel more confident about your learning abilities because of 
this class.” Qualitatively, the most common positive response involved increase in 
self-confidence. 

Strengths: The data collected generated quantitative and qualitative data. 

Weaknesses: Data were incomplete. The collection process was haphazard. Data 
did not seem to be collected from all participants. Class time of 16 hours was too 
short to show significant gains. There was no control group. The sample size for 
each class was small. 



6 




7 



NCSALL Reports #6A 



January 1999 



Appalachian Adult Basic Education Demonstration Center. (1970). 
Demonstration, developmental and research project for programs, materials, 
facilities and educational technology for undereducated adults. Long range 
follow-up of adult education participants: West Virginia state module. Final 
report series. Morehead, KY: Morehead State University. (ERIC Document 
Reproduction Service No. ED 054 411) 

Purpose: The Long Range Follow-Up Survey (LRFUS) had two main objectives: 
(a) to follow a sample of ABE graduates for three years and (b) provide a three- 
year supportive program to reinforce the ABE experience. Inherent in both 
objectives was the intent to create another model (rather than standardized reading 
tests) for assessing impact. The second year of implementation is described here. 

Variables: LRFUS sought to measure the relevance and impact of ABE on the 
economic, family, educational, and community lives of the former students. 

Population: Eighty-five former ABE participants were drawn from rural and 
urban areas of three counties in West Virginia. By the study’s second year, nine 
people were no longer part of the sample. Group I consisted of 30 men who had 
taken part in the original field study and were enrolled in compulsory Title V and 
WIN programs. Group IV consisted of 25 men who were assigned a combination 
of ABE and diesel mechanic class. Groups II and III were 30 female former 
volunteers from Title III ABE. Most of the students from all four groups were 
Level III ABE participants. 

Design: This follow-up study utilized a case-study design. 

Data Collection: The sample population was divided into four groups. Each group 
was followed by a different teacher-counselor. Originally the design called for the 
LRFUS subjects to be compared with a matched control group, but the researchers 
decided against it. 

Instrumentation: There was no instrumentation. However, in supporting the 
LRFUS subjects (one of the objectives), the researchers used a combination of 
newsletters, picnics, and personal contacts. The study used a 1 5-item 
questionnaire to gather follow-up data. 

Findings: This follow up was a description of the second year of a three-year 
study. No comprehensive analysis of results was provided. 
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Strengths: This study had no real strengths other than the objectives of this study, 
which seemed solid. 

Weaknesses : Numerous case studies were provided with no analyses. There was 
no control group. 

Askov, E., Maclay, C., & Bixler, B. (1992). An intergenerational study of the 
impact of computer-assisted instruction with low-literate parents. In T. 
Sticht, M. Beeler, & B. McDonald (Eds.), Intergenerational transfer of 
cognitive skills: Volume I: programs, policy, and research issues (pp. 149-158). 
Norwood, NJ: Ablex Publishing Corp. 

Purpose: Computer-assisted instruction (CAI) courseware was developed by Penn 
State for parents of Chapter I children receiving compensatory education. The 
courseware used a whole-word approach. Its goal was expanded word recognition 
for nonreaders. A research study measured the impact of the courseware on 
parents and their children. 

Variables: Reading achievement was the sole variable studied. 

Population: Parents had to score below a fourth-grade level to be part of the 
sample. Fifty-two completed the required 20 hours of instruction and had pre- and 
posttest data. A control group included 24 subjects. 

Design: This study employed an experimental design. Pre- and posttest scores 
were analyzed using analysis of variance for repeated measures. Courseware was 
used about 80% of the time during instruction. 

Data Collection: Teachers administered the battery of tests upon completion of 
entrance interviews and after 20 hours of instruction. 

Instrumentation: The Slosson Oral Reading Test (SORT), two sections of the 
Baltimore County Design (BCD), and the Bader Reading and Language Inventory 
were used. 

Findings: The scores of the experimental group rose from 3.26 to 3.93 on the 
SORT, from 18.38 to 23.33 on the BCD-E2 (words in isolation), from 20.13 to 
22.67 on the BCD-E3 (words in functional settings), and from 2.33 to 3.63 on the 
Bader Inventory. Significant differences between the experimental and control 
groups were found in all four measures (alpha level at .05). 

Strengths: There was a control group. 
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Weaknesses: The sample size was small. Teachers administered the tests to their 
students. Only one variable was measured. 

Association for Community Based Education. (1989). ACBE evaluations of 
community based literacy programs: 1988-1989. Washington, D.C.: Author. 

Purpose: “To expand knowledge about these literacy programs, produce credible 
evidence of their impacts, identify exemplary models and methods.” 

Variables: A convenience sample of nine community-based adult literacy 
programs was selected. Each program chose the variables for study from a list of 
ten. Possible variables included skills in reading, writing, math, and oral language 
(for ESL); reading and writing activities outside the classroom; fostering 
children’s intellectual activities; community involvement and contributions; self- 
esteem; self-determination; and retention. 

Population: Data were collected from all students in each selected program. The 
sample size varied by program and the variable studied (in most cases the size was 
less than 30). 

Design: Programs were rewarded with a $5,000 mini-grant for participation. The 
design was longitudinal. Baseline data were collected no later than the new 
learners’ third session. Follow-up data were collected after about eight weeks and 
then again after about 30 weeks. At time two, five of the nine programs failed to 
have 65% of the baseline sample present for measurement. One program was 
dropped at time three for lack of a usable sample size. 

Data Collection: Data were collected at the program by trained staff and generally 
took between one and two hours. 

Instrumentation: The Adult Basic Learning Examination (ABLE) was used to 
measure reading and math; the California Achievement Test (CAT) to measure 
writing; and the Basic English Skills Test (BEST) to measure English proficiency 
for ESL learners. The Association for Community-Based Education (ACBE) 
developed a survey instrument to measure reading and writing activities outside 
the classroom, children’s intellectual and academic development, and community 
activities and contributions. The Culture-Free Self Esteem Inventory was used to 
measure self-esteem and an ACBE-developed instrument was used to measure 
self-determination. The same versions of the instruments were administered at the 
two follow-up sessions. 
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Findings: Six programs selected reading skills as a measured variable. Three of 
the six programs showed statistically significant gains. Five programs measured 
writing gains. Only one program showed statistically significant gains at the first 
follow up, which were not sustained at the second follow up. Two programs 
measured math gain. No program showed significant gains in math between the 
first and third data collections. Two programs measured oral English. One 
program showed a statistically significant gain over seven weeks; the other did 
not. 

Learners’ reading activities outside class were measured by eight programs. A 
“ceiling effect” on the baseline data collection precluded meaningful conclusions. 
Learners’ writing activities outside of class were measured for five programs. 
Although there was a ceiling affect, it was less pronounced than for reading. Three 
of five programs showed significant gains. 

Only one program measured children’s intellectual and academic development. 
The moderate gains were not statistically significant. Community activities and 
contributions were measured at two programs. There were significant gains 
between time one and two, but the gains dissipated at time three for one program 
and the numbers were insufficient at the other. 

Self-esteem was measured in all nine programs. Two thirds of the participants 
demonstrated low self-esteem at time one. Seven programs had statistically 
significant gains. Self-determination was measured at seven programs. Three 
programs had significant gains. 

In six programs, 9% of the participants dropped out after two sessions, another 
7% dropped out between six and eleven weeks, and an additional 31% left after 
five months. 

Strengths: The study was carefully designed, and the methods and procedures 
were clearly explained. The measures and instrumentation were solid. 

Weaknesses: There were no control or comparison groups. The small sample sizes 
yielded nonsignificant results for gains that would be significant in larger sample 
sizes. Because programs were permitted to chose the variables studied in their 
programs, some variables were studied in only a small number of programs. There 
was substantial attrition of subjects in the follow-up data collections. 
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Berkovitz, L. (1994). Learning with East Aurora families. Project evaluation. 
Des Plaines, IL: Adult Learning Resource Center. (ERIC Reproduction 
Service No. ED 373 187) 

Purpose: This evaluation assessed Learning with Aurora Families (LEAF), a one- 
year family literacy program developed and implemented by Waubonsee 
Community College. LEAF sought to recruit a minimum of 50 families who had 
four- and five-year-old children enrolled in at-risk programs. LEAF’S objectives 
included increasing the basic and parenting skills of the adults as well as the 
readiness skills of the children. 

Variables: The outcome variables included the impact of LEAF on the adults’ 
basic skills, academic progress, career plans, and the evolvement of their goals 
(personal, educational, economic, and parental). It also ascertained changes in 
parental behaviors. 

Population: LEAF included a total of 162 participants from East Aurora. Fifty- 
two participants were four- and five-year-olds, 5 1 were parents or other significant 
adults, and 59 were siblings. The 5 1 adults included ABE and ESOL students. 

Design: LEAF operated in four-week sessions, each divided into five components: 
(a) adult education, (b) preschool or kindergarten classes, (c) parenting-life skills 
classes, (d) parent-child and family times, and (d) home visitations. The 
evaluation was pre-post comparison. 

Data Collection: Project staff collected the data. Tests and questionnaires were 
administered before intervention and at the conclusion of the project. Data were 
analyzed by an independent evaluator. 

Instrumentation: Reading gains were assessed by administrating the Test of Adult 
Basic Education (TABE). A Likert scale was used to measure parenting goals. A 
family literacy questionnaire had three components: (a) a nine-item chart on 
which parents indicated the frequency of their parenting and readiness behavior 
with their children; (b) a nine-item chart to ascertain the adults’ opinions about 
their children’s education; and (c) four open-ended questions to measure project 
outcomes. The impact of parental behaviors and improvement in family 
relationships were measured qualitatively by home observations as well from the 
questionnaires. 

Findings: The average attendance of basic skills students who began LEAF in 
November was 83 hours out of a possible 132 (62% attendance rate). Those who 
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began in January averaged 83 hours out of a possible 108 (65% attendance rate). 
Thirteen adults of the original 29 (42%) who were given the pretest also took the 
posttest. The reading gain average was .86. One participant took and passed the 
GED test, three were ready to take it, and one enrolled in a community college. 
Although positive impact on parenting behaviors was indicated, the data from the 
questionnaires were not adequately summarized. Pretests of the attitude 
questionnaire about parents’ participation in the education of their children were 
misplaced. However, information from the posttests indicated that parents still felt 
that teachers were better qualified than they were to instruct their children. 
Furthermore, they seemed to view their children—but not themselves— as learners. 

Strengths: LEAF was a thoughtfully designed program which considered the 
whole family. Some of the questionnaires were well designed, yielding both 
quantitative and qualitative data. 

Weaknesses: The sample size was small and no control group was used. Analysis 
of the outcomes was inadequate. 

Boyter-Escalona, M. (1995). Enhancing workers’ skills for the workplace and 
for life. Worker education program final report. Chicago, IL: Northeastern 
Illinois University. 

Purpose: From May 1993 through March 1995, Northeastern Illinois University, 
the Amalgamated Clothing and Textile Workers’ Union, and local businesses 
cooperatively implemented a workplace literacy project. The goal was to provide 
500 workers with literacy skills needed in the workplace through a partnership 
between small businesses, community organizations, a labor union, and a 
university. The project had two components: long-term classes in basic skills and 
job-specific workshops. The project had four levels of program outcomes: worker, 
company, union, and university. Only worker outcomes will be discussed in this 
abstract. 

Variables: Studied were math achievement, English vocabulary and general 
communication skills, self-esteem, participation in union-related activities, 
enhanced worked performance, and opportunities for promotion. 

Population: Four hundred and sixty-five employees participated in the long-term 
classes. The majority were immigrants from Latin American countries, with 
significant numbers from Asia. The short-term workshops had 271 participants. 
The majority were U.S. citizens. 
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Design: Although the report was primarily descriptive, it included a pre-post 
comparison component. 

Data Collection: Data were collected by project staff throughout the project. 
Standardized tests were administered at intake, midpoint, and upon exit. An 
outside evaluator provided a summative evaluation. 

Instrumentation: The TABE measured math achievement; the BEST Oral and 
Literacy Test measured English communication and writing skills; and individual 
learning plans assessed personal goals. Other instruments used to gather 
quantitative data were customized work texts, holistic writing samples, attendance 
records, and company and union records. Qualitative data were collected through 
student evaluations, progress reports generated by teachers, anecdotal comments, 
and open-ended interviews. 

Findings: The average gain in English oral skills was 5.6 points for ESL students. 
The average English writing skills gain was 1 .0. Those who scored highly on the 
BEST Oral test were given the BEST Literacy to measure higher-order language 
skills, averaging a gain of 5.2 points. Twenty students were promoted at their job 
sites. 

Strengths: The project was well designed. Evaluation methods were 
comprehensive. 

Weaknesses: There was no control group. Descriptive analysis of the data was 
weak. 

Bramnick, B. (1981). Assessment of adult basic education program impact. 
Philadelphia, PA: School District of Philadelphia, Division of Adult 
Education. 

Purpose: The impact of ABE was assessed in a sample of classes in 1977-78. 

Variables: The variables were self-reported gain in reading, math, writing and 
consumer skills; enrollment and retention; voting behavior; community 
participation; and employment. 

Population: The sample was selected from 17 classes that met for eight hours per 
week at ten sites in Philadelphia. This population included 1,216 students; 25% 
were randomly selected for the study, excluding students with less than 1 2 hours 
of instruction. The sample size was 267 students. 



NCSALL Reports #6 A 



January 1999 



Design: The design was post only self-report. 

Data Collection: Subjects were surveyed by telephone. The survey included 93 
questions. Approximately one third of the sample was reached; the others could 
not be contacted. The final number was 89. On average 2.6 calls were needed to 
reach the subject and the interview lasted an average of 10 minutes. The time 
period between enrollment in the program and data collection was not reported. 

Instrumentation: Checklist questionnaires were developed specifically for this 
project. 

Findings: Seventy-eight percent of the students reported improved skills in 
reading, 81% in math, 80% in writing, and 50% in consumer skills. Fifty-eight 
percent reported being enrolled in courses other than ABE and 79% planned 
future enrollment. Eight-four percent were registered to vote, with 77% voting in 
1 978 and 84% in 1 979. Fifty-four percent increased community participation, 

47% increased social involvement, 70% felt more confident talking, 33% used the 
library more frequently, 67% read more books, and 52% read more magazines. 
Thirty-five percent of those who had children reported increased attendance in 
PTA meetings, 35% reported increased interaction with their children’s teachers, 
and 70% reported that they helped their children more with homework. Thirty-six 
percent of the students were employed full or part time. Of the employed, 81% 
reported that ABE helped them to get a job, 19% reported a promotion, and 44% 
reported increased job security or pay. 

Strengths: The range of impact variables was wide. A detailed report of methods 
and procedures was provided. 

Weaknesses: The sample size was low. There was no control or comparison 
group. Data were exclusively self-reported. Amount of instruction between 
enrollment and data collection was not reported. 

Brown, E. (1990). Evaluation of the R.O.A.D. to Success program. University 
Park, PA: Institute for the Study of Adult Literacy. 

Purpose: R.O.A.D (Real Opportunities for Advancement and Development) to 
Success was a pilot project developed through a partnership of the Pennsylvania 
Department of Education, the American Federation of State, County and 
Municipal Employees, and the Institute for the Study of Adult Literacy. The goal 
was to improve the reading skills of transportation workers to help them pass the 
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commercial driver’s license (CDL) test. The major components of R.O.A.D. to 
Success were interactive computer courseware and print-based materials. The 
evaluation ran from June 1989 to January 1990. Participant outcome variables 
were measured, and attention was given to the adequacy of curriculum materials 
and the process by which they were developed and used. 

Variables: The outcome variables were reading gains, gains and passing scores on 
the CDL test, and attainment of the CDL. Several variables related to participant 
changes in attitudes regarding the program, materials, CDL test, and being a 
student were also measured. 

Population: The treatment group consisted of workers from the Pennsylvania 
Department of Transportation (PennDOT) who had previously failed the CDL 
exam. Sixty-eight workers attended the classes and 58 completed 100 hours of 
instruction. The average age of the students was 41, with an average reading grade 
of 5.5 and an average of 1 1 years of formal schooling. The sample was selected in 
the following manner: 

• All PennDOT workers from four counties were administered the Quick 
Assessment Test (QAT). 

• Those who scored less than 80 were given the TABE Locator. 

• Those scoring less than 23 (8.6 grade level or lower) were eligible for the 
program. 

• Of 77 eligible workers, 68 enrolled. 

• They were given Criterion References Tests (CRT), which measured basic 
educational skills in context. 

A comparison group was chosen in a similar manner for a control group of 10. 

The comparison group had access to the CDL manual and workbook but not the 
R.O.A.D. materials or classes. 

Design: This study had an experimental design with a comparison group. A chi- 
square test was computed to measure significant differences in workers’ attitudes. 

Data Collection: The QAT was administered by Institute staff. The TABE 
Locator was administered by the Pennsylvania Department of Education and the 
TABE Survey by project counselors. The tests and survey were administered 
before and after the program. 

Instrumentation: The QAT, CRT, and TABE were given to measure basic skill 
levels. An attitude survey was also administered. Various instruments (e.g., 
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observation forms, teacher response forms) were developed to collect formative 
data. 

Findings: The average CRT score increased from 70.9 to 79.8, a significant 
difference. However, change between treatment and comparison groups was not 
significant. On the QAT, the average scores for the treatment group increased 
significantly from 33.4 to 49.7. A significant difference also appeared between the 
scores of the treatment and comparison groups. On the TABE, scores increased 
significantly at the .001 level. Average scores increased from 643 to 700. No 
posttest data on the TABE were available for the comparison group. Although 
shifts were apparent in workers’ attitudes toward the intended objectives of the 
program, most questions showed no significant differences. 

Strengths: This project was adequately designed and evaluated. It had a 
comparison group. Various instruments and methods were used to collect data. 
The data were well analyzed. Scores between tests were correlated. 

Weaknesses: Only a few variables were studied. The sample size was low. The 
comparison group size was significantly smaller than the sample size. 

Claus, R., & Quimer, B. (1995). Saginaw adult basic education: Process and 
product evaluation report 1994/95. Saginaw, MI: Department of Evaluation 
Services. (ERIC Document Reproduction Service No. ED 392 978) 

Purpose: This evaluation of the Saginaw City School System’s ABE programs 
studied the process of instruction and products (outcomes) of that instruction 
during 1994-95. This abstract describes only student achievement and goal 
attainment data. 

Variables: The outcomes component focused on student enrollment, recruitment, 
dropout prevention, student achievements, objective assignment, and attainment. 

Population: The 1,096 students enrolled during the 1994-95 school year were 
classified as handicapped, ESL, incarcerated, senior citizen, and regular. Of the 
latter classification, 629 regular ABE students attended at least 1 2 hours of 
instruction. They were assigned individual objectives. 

Design: This study was pre-post comparison. Data from the 1994-95 school year 
were compared to data from 1992-93 and 1993-94, and focused primarily on the 
assignment and completion of student objectives. 
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Data Collection: Data that described student performance on individually 
assigned goals were recorded by teachers on the Adult Basic Education Reporting 
Form. Data on basic skills achievement were collected by the program supervisors 
and sent to the evaluators. Data were also collected from teacher observations. 

Instrumentation: Reading gains were measured by the TABE. It was not clear at 
what point after 1 2 hours of instruction the posttests were administered. 

Findings: Eighty-seven percent of students who received at least 1 2 hours of 
instruction attained their goals, with 64.8% attaining mastery on over 75% of their 
objectives. (These were objective outcomes for all categories of students.) Four 
hundred and eight of the regular ABE students (64.9%) completed at least 75% of 
their objectives. Compared longitudinally, 449 (74.0%) students attained 75% of 
their goals in 1 992-93 and 1 993-94. One hundred and eighty-eight (22.7%) 
students who received at least 1 2 hours of instruction realized one grade-unit gain. 

Strengths: There were few strengths in the outcomes component of this study. 
However, some of the observational data provided a good indication of what 
actually occurred in the classroom. 

Weaknesses: Attrition was high. There were no rigorous and standardized 
procedures to attain data on reading gains. There was no control group. The 
objectives assigned and attained were not delineated nor categorized in the data 
analyses. Therefore, this data can not be used to make informed conclusions by 
others. 



Connecticut Business and Industry Association. (1992). Connecticut Business 
and Industry Association ’s Statewide Workplace Literacy Project: Final 
performance report Hartford, CT: Author. 

Purpose: This report focused primarily on the goals, objectives, and activities of 
the workplace literacy project. The project centered on nine companies that had 
identified literacy needs as a result of implementing TQM processes. Only the 
outcome data are reported here. 
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Variables: Expected wage gain within six months of completing training and 
tested learning gain were studied. 

Population: The sample consisted of participants in workplace education in nine 
Connecticut companies. 

Design: Companies were asked whether they expected to increase the wages of 
successful learners and, if so, by how much. Seven of the nine companies 
administered posttests to determine learning gain. 

Data Collection: How the tests were administered was not described. The time 
interval between pre- and posttesting ranged from 12 weeks to 36 weeks. 

Instrumentation: There was little information on the tests. They measured 
different competencies for each company and were scored in different ways. 

Findings: Only five companies supplied data on expected wage increases, of 
which two provided reliable data. At one site the expected increase was .52, at the 
other it was .33. Coordinators in neither company were able to stipulate that the 
gain was necessarily attributable to participation in the literacy program. Gain 
scores at five companies were computed for 185 participants (29% of the total). 
Forty-nine percent did not gain at all, 15% gained in the low range (1-20 points), 
22% gained in the middle range (21-40 points), and 15% gained in the high range 
(41-56 points). 

Strengths: The one strength was the study’s workplace context. 

Weaknesses: The sample size was small. There was no established validity or 
reliability of the tests and the tests were idiosyncratic. There was no comparison 
group. The description of methods and procedures was meager. The time interval 
between pre- and posttesting was considerable. 

Coplin, J. (1994). Week Educational Development Program. Final 
performance report Durham, NC: Week & Durham Community College. 

Purpose: This final report described a workplace literacy program conducted at 
Week, Inc. for employees of its manufacturing and warehouse divisions. The job- 
related curriculum was developed from a task analysis. 

Variables: Variables studied were GED completion and learner satisfaction. 
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Population: There was a sample of 8 1 . Twenty-eight participants lacked high 
school diplomas. 

Design: The descriptive design was post only. 

Data Collection: Students were asked about their satisfaction levels. Informal 
interviews were conducted with company staff. Surveys and written comments 
were analyzed. 

Instrumentation: A 19-item satisfaction survey was used. 

Findings: Of the original 36 participants who started in 1991, 1 1 received their 
GEDs. Interviews with managers and program staff indicated that the program 
received support. Students’ written comments suggested that satisfaction was 
high. 

Strengths: None. 

Weaknesses: There was no formal methodology or procedures. This study 
measured students’ satisfaction only. Results of the survey were not reported. 

Coston, C., & Dagard, C. (1995). Champaign County National Workplace 
Literacy Project. Final evaluation report. Urbana, IL: Champaign Board of 
Education. 

Purpose: The Champaign County Workplace Literacy Project was managed 
through the Champaign Board of Education. Four local companies, major 
employers in the county, participated in the project. The project had two overall 
purposes: (a) to provide training to employees based on their needs, and (b) to 
design and implement a curriculum that targeted job- specific skills to prepare 
employees for participating in TQM/SPC processes. The project implemented 
three cycles of classes, each lasting 15 weeks. 

Variables: Variables measured were prose literacy (understanding and using 
information contained in various textual materials) and document literacy 
(processing information found in documents), mathematics achievement, 
knowledge and ability to participate in TQM/SPC processes, and the effects of 
increased worker productivity. 
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Population: All participants were employees of four local companies. They were 
selected from various job classes, including maintenance, technical, 
manufacturing, and administration. Two hundred and fifty-five employees 
participated in the classes. Standardized pre- and posttest prose achievement data 
were collected from 161 individuals, document achievement data were collected 
from 171 , and math achievement data from 166 . 

Design: An external evaluator used the Triphase Evaluation process based on 
Stuffelbean’s decision-making model. The three phases evaluated were input, 
process, and outcome. The analysis involved a comparison of matched pairs of 
the pre- and posttest results and a percentage analysis of the observed change 
during treatment. 

Data Collection: Project staff collected data from most sites. The project 
evaluator analyzed the statistics. No posttest data were collected from two sites. 
The use of the ABLE was discontinued after the first cycle. Problems arose in 
collecting interview data from some sites and during some cycles. 

Instrumentation: Instruments included participant and supervisor surveys, 
supervisor interview forms, and two standardized tests (TALS and ABLE). 

Findings: Data were inconclusive on the extent to which this project impacted 
workers’ levels of productivity and their understanding and use of SPC and TQM. 
The aggregate result of prose literacy for all participants demonstrated a 
statistically significant growth during all cycles, although achievement in 
document literacy showed no significant growth. In fact, participants experienced 
a decrease in the average scores on the latter. Most participants experienced 
statistically significant increases in their math scores. 

Strengths: Evaluation methods were comprehensive. 

Weaknesses: There was no control group. Data collection procedures were not 
rigorous. Much of the data were incomplete. The standardized test did not seem to 
measure what was taught. 

Cox. L., & Lane. C. (1981). Evaluation of the adaptation of the Personalized 
System of Instruction to nontraditional adult learners. Final report. Volume 
III: PSI meets ABE. Durham, NC: Research Triangle Institute. (ERIC 
Document Reproduction Service No. ED 208 181) 
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Purpose: The Personalized System of Instruction (PSI) was a self-paced, mastery- 
oriented system that stressed the use of printed instructional material, immediate 
feedback, and peer proctors. This method had been successful in teaching college 
students. The study sought to gauge the effectiveness of PSI on adults in settings 
other than colleges and universities. The main objective was to answer the 
question, “Is further investigation of the use of PSI for nontraditional adult 
learners likely to be worthwhile?” The study was also designed to identify salient 
areas for further investigation. Some outcome data were generated. 

Variables: Outcome variables were reading and mathematics achievement; 
changes in self-concept, self-dependence, and self-assertiveness; and evidence of 
increase in life-coping skills. 

Population: All participants were ABE students. They were enrolled in classes 
meeting twice a week for two- or three-hour sessions. One class was situated in a 
housing authority site, one was at a community school setting, and one in a prison. 
Most of the students were Black and approximately 90% were female, except for 
the prison inmates who were all males. Students ranged in age from 1 8 to 50, but 
most were 30 to 50 years old. 

Design: The research design was a case-study approach. Although the primary 
emphasis was on collecting observational data from three sites during a six-month 
implementation, the researchers also collected impact data. They collected data 
from two other sites for comparison. 

Data Collection: To insure baseline data, students in the three research sites and 
two comparison sites were given the ABLE and self-concept scale within the first 
four weeks of class. Classroom teachers administered the instruments. Pre- and 
posttest ABLE data were recorded for ten PSI students and five from the 
comparison groups. Twelve PSI students and nine comparison group students had 
pre- and posttest scores on the self-concept scale. 

Classroom observations were to be conducted primarily by the classroom teacher, 
while researchers planned to visit each PSI site once a week. Observational data 
were summarized by site. Data collected were the numbers of units mastered, pre- 
and posttest achievement and self- concept scores, and attendance records. 

Problems occurred with the interview process. Interviewers spent a lot of time 
“tracking down” interviewees. There were 319 incomplete interviews: 70% were 
due to the inability to locate interviewee, 11% were unwilling to participate, and 
7% provided very little useful information due to physical, mental, or emotional 
disabilities. 
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Instrumentation: Researchers developed several forms to collect observational 
data. The ABLE was used to collect reading and math achievement data. Self- 
esteem data were obtained by the Tennessee Self-Concept Scale. 

Findings: The PSI students showed average gains of almost one grade level in 
reading and more than one grade level in math. No such gains were found in the 
comparison groups. However, the latter group’s entry-level scores were much 
lower than the PSI groups; thus, they did not represent the same population. Self- 
concept pretest scores were almost identical for the two groups. PSI students 
showed a slight increase at posttest while the comparison groups showed a slight 
decrease. 

Strengths: There was a comparison group. 

Weaknesses: No real conclusions could be reached regarding PSI effect on self- 
concept because the sample size was so small. The comparison group had a much 
smaller size than the treatment group and was not representative of the treatment 
group in all variables measured. The research design and data collection were 
weak. 

Darkenwald, G. (1983). Project F.I.S.T.: Functional In-Service Training. 
Submission prepared for the Joint Dissemination Review Panel. Edison, NJ: 
Middlesex County College. 

Purpose: Project F.I.S.T. was a volunteer-based administrative and instructional 
delivery system aimed at meeting the special needs of low-level readers (0-4). 
F.I.S.T. evaluated its program from 1981 through 1983 to apply for validation by 
the Joint Dissemination Review Panel (JDRP). It measured tutor and student 
outcomes. Only the latter is discussed in this abstract. 

Variables: Student outcomes measured were reading achievement and change in 
reading behaviors. 

Population: The sample size for both the initial and replication group was 26. All 
the ABE students lived in New Jersey. The mean age for both groups was 32.5; 
mean grade completed was 9.2. Fifty-three percent were women. Half were from 
minority backgrounds (nearly all Black) and 68% were employed. 

Design: A single group pre-post design with replication was employed. 
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Data Collection: Data were collected by project staff under the supervision of an 
external evaluator who also performed the statistical analyses. The standardized 
reading test was administered at entry and at four-month intervals. 

Instrumentation: The Woodcock Reading Mastery Test was used to measure 
reading gains. A questionnaire was administered in both studies to gather data on 
the application of reading skills and other positive life changes due to 
participation in F.I.S.T. 

Findings: In the original study, both the initial and replication groups showed 
statistically significant gains on the Woodcock. The gains for the replication 
group were similar to those of another JDRP-approved program, the Jefferson 
County Adult Reading Project (JCARP). The initial group performed considerably 
better than the replication and JCARP groups. 

All but one of the learners indicated positive life changes and general 
improvement in reading skills. All 21 items on the questionnaire showed gains. 
Especially significant was that one fifth of those looking for jobs became 
employed, nearly three fifths started reading newspapers for the first time, and 
about a third of the parents began helping their children with schoolwork. 

Strengths: The replication design was rigorous. Methodology was well described. 
Results were compared to another nationally validated program. 

Weaknesses: There were low numbers and no control group. The data collection 
methods for the questionnaire were not well described and it was unclear how it 
was developed or by whom. Some data were self-reported. 

Darling, S. (1982). Jefferson County Adult Reading Program. Submission 
prepared for the Joint Dissemination Review Panel. Louisville, KY: Jefferson 
County Adult Reading Program, Jefferson County Board of Education. 

Purpose: The Jefferson County Adult Reading Program (JCARP) was initiated in 
1978 to develop and field test a model adult literacy program. Field testing in 
1978-79 provided information to help redefine the model for the 1979-80 funding 
year. Two years of field testing showed that teachers’ certification had little 
impact on students’ progress in reading; materials should be of high interest to 
adults but designed for those reading below a sixth-grade level; and student 
reading gains were related to increased hours of instruction. Essential to the 
subsequent model was the idea that “student-staff interaction would result in 
consistent attendance and high retention.” As a consequence, JCARP integrated a 
counseling construct into most phases of its program, including student 
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recruitment, staff training, instructional design, and evaluation. This study was 
submitted to the Joint Dissemination Review Panel and described data from the 
field test years as well as the 1 980-8 1 model program. 

Variables: The outcome variables were reading gains and attrition rates. Hours of 
instruction completed were also studied, along with the achievement of student 
goals (such as employment and education) in 1980-81. 

Population: The initial field-test year (1978-79) included 293 students who tested 
below 4.0 grade equivalent on the SORT. The following year studied 508 students 
at 26 locations reading up to a 6.0 grade equivalent. During the 1980-81 model 
year, 591 students enrolled. Of the 1,392 students enrolled, 54% were male and 
46% were female; 59.8% were Black and 36% White. Other groups enrolled were 
categorized as Hispanic, Asian or Pacific Islander, American Indian, or Other. 
Thirty-nine percent of the students were employed. The median age was 27. 
Students were considered enrolled once they took a pretest. Separation from a 
program occurred after one month of nonattendance or when students indicated 
their withdrawal. Demographic distributions of the JCARP students were similar 
to other ABE students from Kentucky. 

JCARP’s attrition rates in 1980-81 were compared with comparable programs in 
Kentucky. Five hundred and ninety-one learners enrolled in JCARP that year and 
706 in 10 comparable programs. 

Design: JCARP used an experimental design. 

Data Collection: Demographic and attrition data were collected from all 10 ABE 
centers. Data to compare reading gains were collected from five programs located 
in urban environments comparable to Jefferson County. 

Instrumentation: The TABE (levels E and M) was used to measure reading gains 
during the comparison year. TABE raw scores were converted to grade equivalent 
scores. JCARP had a 6.0 grade-equivalent level as a maximum for enrollment, so 
the SORT was used as a screening instrument. Comparison groups had no such 
guidelines and screening tests were not used. 

During the two field-test years, JCARP used different standardized reading tests 
to measure reading gains. In 1978-79, the Woodcock Reading Mastery Test was 
utilized and in 1979-80, the Adult Basic Learning Examination (ABLE). 
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Findings: At 22%, JCARP participants experienced a lower attrition rate than 
comparable programs. Among the comparison groups, the highest attrition rate 
was 80% and the lowest was 52%. A chi-square analysis was significant at the 
.001 probability level. 

JCARP students made significant gains in reading ability. The average gain was 
1.53. A t test of these results was significant at the .001 probability level. The 
mean hours of instruction was 82. 

The gains of the JCARP participants were compared with those from five other 
learning centers (173 matched subjects). JCARP students showed a .70 greater 
equivalent gain than did comparison group participants. 

A t test on hours of instruction was not statistically significant. 

Two hundred and eighty-five students had employment-related goals; 139 
achieved them. Eighty-eight JCARP participants of the 325 who listed high school 
equivalency as a goal made sufficient progress to enroll in GED programs. 
Furthermore, 62 students enrolled in other education or training programs and 175 
students said they planned to continue their education. 

Strengths: The study’s experimental design included a comparison group. The 
design was rigorous so it ensured statistical reliability and generalizability. This 
research included data from a two-year pilot study. 

Weaknesses: Although common practice in Kentucky, TABE standardized testing 
procedures were violated because no time limitations were imposed. Furthermore, 
comparison data were used only during 1980-81. Because teachers tested their 
own students in the JCARP program and the comparison programs, tester bias 
could be a factor making the findings less reliable. 

Darling, S., & Hayes, A. (1989). Breaking the cycle of illiteracy: The Kenan 
Family Literacy Program model. Final report 1988-89. Louisville, KY: 
National Center for Family Literacy. 

Purpose: This is the 1988-89 final report of the Kenan Trust Family Literacy 
Project implemented by the National Center for Family Literacy (NCFL). Seven 
pilot sites— three in Kentucky and four in North Carolina-implemented the Kenan 
model. It required basic skills instruction for children and adults in the program. 
The major goal of the project was to “break the cycle of illiteracy” by integrating 
early childhood education with basic education and instruction in parenting skills 
for the parents. 
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Variables: NCFL and its researchers developed summary profiles of types of 
adults who enrolled in their programs. The following variables described the 
differences among groups: (a) degree of acceptance of social norms, (b) 
commitment to change, (c) hopefulness of change in self and personal conditions, 
(d) confidence of change, (e) personal capability to learn and change, and (f) 
supportiveness of environment. 

Achievement variables were improvement of parenting skills, increases in home 
literacy, math and reading achievement, GED attainment, and pursuit of 
employment or further education. 

Population: The sample size for the pilot sites was 1 14 adults and 288 children. 
The average age of the parent was 26.6 years. The families averaged 2.7 children. 

Design: Data were descriptive with a pre-post component. No systematic research 
design was evident. 

Data Collection: The sites used various procedures and instrumentation to collect 
data. The report was not clear in describing which procedures were implemented 
nor when and how they were used. 

Instrumentation: Instruments included tests developed by teachers, case studies of 
families, anecdotal and descriptive records, the TABE, interviews, surveys, and 
observations. 

Findings: The following “types” of parents received GED certificates: Type A— 3, 
Type B-4, Type C— 1, Type D-l, Type E-2, Type F-2, and Type G-2. Fifty- 
nine percent gained over one grade level and 29% gained over two grade levels on 
the TABE Reading test. Seventy-one percent gained one grade level, 48% gained 
over two grade levels, and 24% gained three grade levels on the TABE Math test. 
In addition, 80% gained over one grade level, 45% gained over two grade levels, 
and 45% gained over three grade levels on the TABE Language test. 

Strengths: Comprehensive variables were measured. 

Weaknesses: There was no comparison group. There were no details on 
methodology. The report failed to provide details on context. Except for tested 
learning gain, most data appeared to be self-reported. 

Darling, S., & Hayes, A. (1996). The power of family literacy. Louisville, KY: 
National Center for Family Literacy. 
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Purpose: Beginning with the 1991-92 school year and continuing through 1994- 
95, NCFL implemented the Toyota Families for Learning Program and the Knight 
Family Education Program at 80 sites in 19 cities. This report summarized the 
research findings during those funding years as well as the outcomes of follow-up 
studies on families from one to six years after enrollment. 

Variables: The outcome variables measured for the adults were retention rates, 
language and math gains, GED attainment, self-esteem, confidence in parenting, 
and amount of home literacy activities. The follow-up studies addressed obtaining 
and keeping employment, continued enrollment in education or training programs, 
reduction of dependence on public assistance, and parental involvement in their 
children’s school. 

Population: Almost 2,500 families participated during the four funding years. It 
was not clear how many adults participated. The follow-up studies involved 200 
adults and 315 children who had attended the original Kenan model sites, the 
Hawaii Pilot Family Literacy program sites, or those sponsored by NCFL in 
Florida, New York, and Arizona. Adults in the NCFL programs received a median 
of 140 hours of instruction over a 24- week period. 

Design: The data were primarily descriptive. A systematic research design was 
not evident. 

Data Collection: Follow-up data were collected by telephone or face-to-face 
interviews. Other procedures were not described. 

Instrumentation: Academic achievement instrumentation included CASAS and 
the TABE. The Rosenberg Self-Esteem Questionnaire measured general self- 
esteem. It was not clear how the other variables were measured. 

Findings: In the ongoing studies, adults at the lowest levels of literacy at 
enrollment gained an average of 4.4 CASAS scale points. Twenty-eight percent 
made gains of one or more CASAS levels. The average gain on the TABE was 1.3 
grade levels. Descriptive data also indicated that parenting skills, parent-child 
activities, and adult self-esteem improved. 

Of the 200 parents included in the follow-up studies, 51% attained the GED, 43% 
were employed (compared with 14% before enrollment), 13% enrolled in higher 
education or training programs, 1 1% were continuing with ABE, and 23% of 
those on public assistance at enrollment had ceased to collect benefits. 
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Strengths: Comprehensive variables were measured. Much of the data were 
compared to literacy programs that focused solely on adults. 

Weaknesses: There was no control group. The report failed to provide details on 
context or methodology. Other than tested learning gain, most data appeared to be 
self-reported. 

Educon Diversified, Inc. (1995). Investing in Minnesota’s future: An analysis 
of economic, societal, and personal impacts of literacy and adult basic 
education programs in Minnesota. Belle Plain, MN: Author. 

Purpose: The purpose of this report was to inform the governor, the Minnesota 
legislature, and state taxpayers on the outcomes of adult basic education. 

Variables: The variables primarily concerned data on economic impact. 

Population: This report wove an argument for impact using data from a variety of 
sources. The Minnesota Department of Education supplied most of the statistics 
(presumably the same data reported to OVAE each year). The sample size varied 
with the statistic presented. 

Design: This purely descriptive report used data from the state, the popular press, 
NALS, and other sources to make its case. Some anecdotal testimonials were 
included. 

Data Collection: These procedures were not described. 

Instrumentation: The instrumentation was not described. 

Findings: Between 1960 and 1994, 147,000 persons received their GED. Most of 
the data were used to demonstrate the need for adult literacy education rather than 
to portray outcomes. 

Strengths: The report presented a wealth of useful descriptive information. 

Weaknesses: Although the report claimed to be an outcome assessment, it was 
actually more of a public relations document. No data on outcomes were actually 
collected; all sources were secondary. Most of the statistical data were presented 
to argue need rather than to demonstrate impact. The methods and procedures 
used to collect the data were not described. 
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Fingeret, H., & Danin, S. (1991). They really put a hurtin’ on my brain: 
Learning in Literacy Volunteers of New York City. Durham, NC: Literacy 
South. (ERIC Document Reproduction Service No. ED 332 026) 

Purpose: This research study evaluated an educational program, Literacy 
Volunteers of New York City (LVNYC), where tutors provided small group 
instruction to adults with low literacy skills. The major question addressed was 
the impact of Literacy Volunteers of New York City on its students. 

Variables: Change was examined in three areas: (a) literacy skills; (b) self- 
concept, attitudes, and beliefs related to literacy development; and (c) 
involvement in literacy tasks outside the program. A quantitative piece analyzed 
students’ standardized test scores and writing samples. Writing variables included 
sub-skills such as the number of words used in general and per sentence, content 
organization, vocabulary, sentence and paragraph structure, and mechanics. 

Population: Twenty tutors and 16 LVNYC staff members were interviewed. 
Sixty-five students were interviewed in focus groups and another 37 by telephone. 
Ten of the focus group students were interviewed individually several times as 
case studies. The reading scores of 1 14 students and 66 writing samples from 33 
students were analyzed. 

Design: This was a qualitative case study, although test data were analyzed at 50- 
hour intervals. 

Data Collection: Fieldwork was conducted and data collected over seven months 
in 1990. Open-ended focus group interviews, individual interviews, and 
observations were the primary methods used to collect data. Coding the 
qualitative data involved using a computer text- analysis program specifically 
developed for the project. The data to analyze students’ writing samples were 
collected at various points. The first writing sample was collected during the first 
month of participation. Standardized reading test data were collected at 50-hour 
intervals. 

Instrumentation: A consultant generated a rubric modeled on various writing 
assessment tools to analyze the writing samples. A reading vocabulary inventory 
was given as a locator when students enrolled in LVNYC. Those who scored 
below 3.0 were administered the Test of Adult Basic Education (TABE). Those 
who scored below 3.0 were given the Adult Basic Learning Examination (ABLE). 
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Findings: Writing Samples : The relationship between cumulative length of time 
in the program and change in writing scores was examined. Thirty-three paired 
writing samples representing three groups were analyzed. The mean length of time 
in the program for each group was 5.38 months, 9.43 months, and 16.67 months, 
respectively. The difference score average for each group was 9.23, 6.75, and 7.5, 
respectively. Although the researchers concluded that rapid gains in writing scores 
occurred in the initial stages of instruction and then reached a plateau, they were 
not sure what these findings meant. They hypothesized that slower writing gains 
for students over time may signify changes in content from personal stories to 
varied subject matter. No claims were made about the statistical significance of 
these findings. 

Reading Gains : The researchers hypothesized that the more instructional hours a 
student received, the greater the gain in reading grade level. Average grade-level 
score at entry for 1 14 students was 1.518. Average posttest score was 2.4. 

Average grade-level gain was .9 years. Average length of time in the program was 
229 hours. The gains were not found to be statistically significant. The researchers 
noted that “the results of the Analysis of Covariance (ANCOVA) reveal no 
statistically significant effects of differences in grade level gains between the pre 
and posttest scores based on length of time in the programs, grade level at entry 
into the program, or the relationship between the length of time in the program 
and entry grade level score.” 

To assess qualitative impact of the program on LVNYC’s students, the 
researchers presented an impact framework that viewed literacy as “practices” 
(i.e., something that one participates in doing rather than being a characteristic of 
the person). They concluded that literacy is “situational” and distinguished 
between private and public literacy practices. Public and private literacy practices 
occurred inside and outside LVNYC. 

LVNYC students assessed their reading and writing skill development by offering 
examples of what they could do during their tutoring sessions. They also assessed 
their progress by noting their progression through books of increasing difficulty, 
and gauging how their reading speed and comprehension had improved. The 
positive impact of LVNYC on some of its students was seen in the high level of 
involvement they had in extracurricular activities offered by the program, 
including participation in tutor and student orientation and training, state and 
national conferences, a publishing program, and student committees. 

The impact of new “private” literacy practices outside the program was most 
evident in activities that had a high level of predictability (e.g., paying bills). 
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Interpersonal relationships also changed, such as students doing tasks 
independently or involving new people to support their reading and writing 
development. Four environments-jobs, banks, public transportation, and grocery 
stores— were consistently mentioned by students as places where they engaged in 
public literacy practices in new ways. The researchers also found that students’ 
beliefs and attitudes toward reading and writing tasks changed as did their self- 
image as learners. 

Strengths: A rigorous design included qualitative and quantitative data. The 
framework generated by the qualitative data— that literacy is practiced 
situationally— is important to the field. 

Weaknesses: The small sample size for some of the variables examined make it 
difficult to conclude that the results were representative of the program as a 
whole. There was no control group. 

Hayes, A. (1993). Hawaii Family Literacy Pilot Project: Summary report 1990- 
93 and interim follow-up. Louisville, KY: National Center for Family 
Literacy. 

Purpose: In 1989, three sites in Hawaii adopted and adapted the Kenan Family 
Literacy Program model. The report evaluated the project after a three-year period. 
This abstract will describe only the impact upon adults. 

Variables: Measurement of academic, social, and personal characteristics of 
adults was planned, but very little pre- or posttest data were obtained. 

Population: One hundred adults initially enrolled in one of the three sites. 
Seventy-three remained until the end of the first year. 

Design: The design was descriptive. 

Data Collection: Two sites implemented the Hawaii Competency-Based High 
School Program and the third a GED-enrichment program. Little description of 
collection procedures was included. 

Findings: Thirty four of the 41 parents who did not have a diploma at enrollment 
received GED certification. Of the adults who had pre- and posttest achievement 
data (the numbers were not stated), all improved approximately one grade level 
during the time enrolled. At one site, 29 or 30 students attended a special 



31 




32 



NCSALL Reports #6A 



January 1999 



summer/transition program at a local community college. There were few changes 
in employment. 

Strengths: There were no real strengths. 

Weaknesses: There was no control or comparison group. There were few details 
on methodology. The data on adults were self-reported. 

Hudson, P., & Gretes, J. (1994). New Horizons: A national workplace literacy 
program. Final report and external evaluation impact study. Conway, SC: 
Horry-Georgetown Technical College. 

Purpose: New Horizons was a workplace literacy project in which employees of 
Georgetown Steel Corporation attended classes provided by Horry-Georgetown 
Technical College. This project had eight objectives. Among them were to (a) 
conduct a comprehensive needs assessment of employees and develop 
competencies bases on those needs, (b) design course curriculum around those 
competencies, and (c) develop an individual education plan (IEP) for each 
employee who participated in workplace literacy classes. 

Variables: The variables were reading, writing, and math gains; self-confidence as 
readers, writers, and in mathematics; self-esteem; and on-the-job problem solving, 
time management, and productivity. 

Population: Fifty-three percent (N- 454) of Georgetown Steel employees 
participated in at least one of the classes. Their mean age was 40 years old. Most 
participants were male. Two hundred and ninety-four employees were White and 
159 were Black. The majority of participants had been employed by the company 
for more than six years. However, only 57 participants had pre- and posttests in 
math, 61 in reading, and 19 in writing. It is not clear how many completed the 
surveys to gather data for the other variables. 

Design: There was a pre-post comparison design. 

Data Collection: Besides pre- and posttesting, interviews were conducted with 
participants and their supervisors. Researchers also analyzed IEPs. Supervisors 
were interviewed each quarter using information from the survey. 

Instrumentation: The Bader Reading Inventory was used to place those who read 
above 4.0 in reading and writing classes. Those who scored lower were placed in 
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an emergent reading class. Customized tests were used to measure reading, 
writing, and math gains. 

A project-developed survey was used to gather information from supervisors. It 
consisted of open-ended questions. A 12-item participant survey was also created, 
with a five-point scale ranging from “strongly agree” to “strongly disagree.” 

Findings: Participants had a 39% increase in math, a 22% increase in reading, and 
a 15% increase in writing. In addition, 91% of the participants “strongly agreed” 
or “agreed” that their communication skills had increased, 77% believed the 
program had increased their productivity, and 93% felt their self-esteem had 
improved. 

Strengths: Many variables were measured. 

Weaknesses: The sample size seemed small, although the true sample size was not 
made clear. There was no control group. Much of the data were self-reported. 

Data collection methods were not well described. 

Koen, S. & Musumeci, M. (1984). BES Adult Literacy Project Norwalk, CT: 
MATRICES Consulting Group. 

Purpose: The BES Adult Literacy Project was the classroom instruction 
component of Bronx Educational Services. This research study was its submission 
to the Joint Dissemination Review Panel and described student outcome data from 
1981 to 1984. 

Variables: The outcome variables were reading gains and attrition rates. 

Population: There were seven cohorts of students. All adults were in the 
nonreader or beginner reader category. Ninety students had matched pre- and 
posttest scores, which represented 52% of the entire BES enrollment during the 
three years of program evaluation. The majority of the students (81%) were Black; 
16% were Hispanic. Fifty-nine percent were males. Three quarters of the students 
were employed; the majority earned less than $12,000 a year. 

Design: A quasi-experimental research design, the Cohort Design, was used to 
assess program impact. It combined longitudinal and cross-sectional approaches 
into one research plan. T tests were computed for the entire student sample and for 
individual cohorts. An independent t test was also used to determine the 
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difference in reading achievement between students exposed to the BES program 
and those who were not. 

Data Collection: Students were tested at the beginning and the end of each 
instructional cycle (100 hours). TABE tests were administered by BES staff. The 
project evaluator spot-checked scoring for accuracy. 

Instrumentation: The TABE (levels E and M) was used to measure reading gains 
during the comparison year. TABE raw scores were converted to scaled scores 
and grade equivalent scores. Scores were transferred to data rosters provided to 
external evaluators. 

Findings: Reading gains on the TABE were statistically significant for all seven 
cohorts. Cross-sectional comparisons demonstrated on all TABE reading scales 
that BES students outperformed comparable groups of students not exposed to the 
program. 

Strengths: The study used a quasi-experimental design which included a control 
group. The design was rigorous. Reading gains were measured over time. 

Weaknesses: Very few variables were measured. The uniqueness of the program 
makes it difficult to duplicate. 

Lafayette School Corporation. (1991). Lafayette Adult Reading Academy 
(LARA) and St Elizabeth Hospital Employee Literacy Program. Final program 
report. Lafayette, IN: Author & St. Elizabeth Hospital. 

Purpose: This literacy project sought to increase the usefulness of health-care 
materials by reducing their volume and to improve the basic skills of hospital 
employees. 

Variables: Outcome variables measured were literacy, verbal communication, and 
problem solving around particular work tasks or strands, such as terminal 
cleaning, safety instruction, and disaster codes. 

Population: Two hundred and thirty-six participants, primarily white females 
employed at St. Elizabeth Hospital, participated in the project. The majority had 
been employed at the hospital for five years or less. They were drawn from 
various departments. 
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Design: The report was primarily descriptive. This was not a research study, but a 
literacy project in which nine training strands were developed. 

Data Collection: Employee reactions were elicited through surveys, interviews, 
and testimonials. Achievement gains were measured by pre- and posttests of job- 
specific skills. Change in participant behavior on the job was measured by 
observation, checklists, and rating scales. After a comprehensive needs analysis, 
new types of health-care forms were created to measure progress. It was not clear 
who developed them or how. 

Instrumentation: All instruments were generated by the project. 

Findings: All participants tested higher on targeted skills. 

Strengths: Variables measured were job specific. Useful forms were developed for 
the hospital. 

Weaknesses: There was no comparison group. Data were exclusively self- 
reported. It was not clear how variables were measured. Instrumentation was 
developed entirely by the project and not well described. 

Lashof, J. (1992). BEST: A learner-centered workplace literacy partnership of 
the Vermont Institute for Reliance and General Electric Aircraft Engines. 
Rutland, VT: Vermont Institute for Self-Reliance. 

Purpose: From April 1990 to March 1992, the Vermont Institute for Self-Reliance 
conducted context-based literacy instruction for employees of two General 
Electric Aircraft Engine plants and two Burlington Electric Department plants. 
Responsive Text (RT), a computer-augmented reading environment, was one of 
the primary modes of instruction. One of the project’s goals was to evaluate the 
extent to which RT provided an effective tool for developing employees’ literacy 
skills. The project also sought to increase the literacy levels of employees and 
evaluate the extent to which these increases improved their work performance. 

Variables: Instructors’ perceptions of improvement in employees’ literacy skills 
were measured. The variables included comprehension of expository, work, and 
narrative materials; decoding ability; vocabulary knowledge; interest in reading 
and writing; participation in group activities; ability to use and interest in RT; and 
writing ability. 
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Supervisors’ perceptions of improvement in employees’ literacy skills were 
measured. Variables included changes in the employee’s ability to read and write 
job-related materials, and the frequency he/she reads and writes at work. 
Participants’ perceptions of these categories were also measured. 

Participants’ perceptions of changes in their work performance were measured. 
The variables included leadership in the workplace, initiative in the workplace, 
knowledge of her/his job, self-confidence in the workplace, absenteeism, quality 
of work, and level of responsibility. Supervisors’ perceptions of these categories 
were measured. 

Population: It is not clear how many workers actually participated, because the 
numbers reported are a duplicated count. In other words, participants were 
counted more than once. 

Design: This was an evaluative report. There were pre-post comparison 
components and a portfolio assessment. 

Data Collection: Qualitative and quantitative data were collected. Multiple 
sources of data were used to triangulate consistent patterns of changes. 

Instrumentation: Portfolio assessment procedures were used to measure 
outcomes. The instruments included pre- and post-interview forms, reading 
assessments, writing samples, a scale of teachers’ perceptions of improvement, a 
participant self-evaluation form, a supervisor’s response form, and a responsive 
text evaluation form. All instruments were developed by the project. They drew 
on the holistic assessment approach developed by Susan Lytle and incorporated a 
portion of the California Adult Learner’s Progress Evaluation Program. 

Findings: Participants increased in each of the four measures of literacy 
performance. In all but two cases, the mean differences were statistically 
significant. Instructors and supervisors perceived participants as making gains in 
the four areas of literacy performance as did the participants themselves. 
Participants and supervisors also perceived major changes in work performance. 

Strengths: Assessment instruments were very comprehensive. Multiple sources of 
data were used to triangulate consistent patterns of changes. 

Weaknesses: Much of the data were perceptions reported by instructors and 
supervisors. There was no control group. The number of employees served is 
reported as a duplicated count. It is, therefore, not clear what the actual n is. 
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Lipiec, J. (1993). Verifying applied literacy skills (VALS) in ABE Programs. 
Schnecksville, PA: Lehigh County Community College. 

Purpose: This study compared the effectiveness of a traditional GED curriculum 
with one based on applied literacy skills. It also sought to determine the validity of 
the Educational Testing Service’s Tests of Applied Literacy Skills (TALS) as a 
predictor for GED performance. 

Variables: Learner outcome variables were GED attainment and performance, 
reading and math achievement, retention, and meeting stated goals. 

Population: Originally 70 students enrolled in one of the two classes, but only 14 
students from each class finished the program and had complete pre- and posttest 
data. Thirty-five students were reached by telephone and interviewed three months 
after completing the program. 

Design: This study had an experimental design with a control group. Pre- and 
posttests were compared using a t test on mean scores. 

Instrumentation: The TALS and ABLE were used for achievement data. Results 
of GED practice tests were also analyzed. A seven-question follow-up survey was 
used. 

Data Collection: Pre- and posttests were administered by project staff at the 
beginning and conclusion of the classes. GED practice tests were administered 
during class. Follow-up interviews were conducted with learners three months 
after they had completed the program. Retention information was collected by 
attendance sheets signed by students at each session. The research staff of the 
college conducted the statistical analyses. 

Findings: Approximately 40% of the students from each group completed the 
program. This result did not support the original hypothesis regarding retention. In 
fact, the control group students averaged 1 .5 more sessions than the experimental 
group. The average age of those who completed the GED program was 39 years, 
compared to 27 years for those who dropped out. 

Both groups showed significant gains on the TALS and ABLE. However, the 
experimental group gain was more significant in traditional basic skills while the 
control group’s gain was more significant in applied basic skills. Again, this result 
countered one of the original hypothesis of the study. When the data from the two 
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groups were compared, no significant score differences appeared on the ABLE, 
TALS, or GED practice tests. 

The follow-up interviews revealed that the amount of progress made by 
experimental and control groups in meeting personal goals was essentially the 
same. 

Strengths: There was a control group. Appropriate variables were measured. 
Follow up was conducted. The researchers asked questions which are important to 
the field. Local data were compared to national data. 

Weaknesses: There were small sample sizes in each group and substantial 
attrition. The data collection methods could have been better described. 

Lipscomb, J., & Cronin, C. (1994). Tools for the 21st century. Southwest 
Alabama Cooperative Literacy Project. Final evaluation. Mobile, AL: 
University of South Alabama. 

Purpose: Seven manufacturing plants and the University of South Alabama 
developed a workplace literacy program, the Southwest Alabama Cooperative 
Literacy Project. The primary objective was to increase productivity by increasing 
the literacy skills of employees. The project lasted 18 months in 1993-94. 

Variables: Reading and math gains were the two variables studied. 

Population: Twenty-six employees participated in one or more sessions of reading 
classes and 35 in one or more of the math classes. The majority were males 
between the ages of 35 and 60. They held diverse positions at their manufacturing 
plants. All scored below seventh grade on the ABLE. 

Design: Data were primarily descriptive. There was a pre-post component. 

Data Collection: Data collection methods were not adequately described. 

Instrumentation: The ABLE was used to measure reading and math achievement. 

Findings: There were 20 pre- and posttests. The average gain was 4.2 points on 
the ABLE, Form C. 

Strengths: There were no strengths. 
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Weaknesses: Sample size was small. It lacked a control group. Data were 
primarily anecdotal. No correlation was made between literacy gains and job 
performance. This was not a formal study. 

Mikulecky, L., & Lloyd, P. (1993). The impact of workplace literacy programs: 
A new model for evaluating the impact of workplace literacy programs. 
Bloomington, IN: Indiana University, School of Education. 

Purpose: Because many evaluations of workplace literacy programs have been 
superficial, NCAL funded a project in 1990 to develop and pilot a model for 
evaluating the impact of these programs. This study produced data on the impact 
of programs at two sites. An ancillary goal was to refine the model for use at other 
sites. 

Population: Participants were employees enrolled in workplace literacy programs 
at two companies with very distinct work sites. Each company had education 
programs which were recognized as effective by state and federal agencies. 

Delco Chassis. Rochester. NY 

Delco Chassis was a unionized electrical motor manufacturing plant with over 
3,600 employees. Education programs were operated jointly by union and 
management with support from state and regional agencies. All learners in this 
study were enrolled in three different types of programs, (a) The technical 
preparation class had 14 participants. This class was a six-week, full-time course 
designed to prepare them for further training. A control group of 12-15 employees 
had enrolled but not yet begun the class, (b) The GED preparation class had 15 
participants. No control group was described, (c) The ESL class had 15 
participants. A control sample contained five. 

Cumberland Hardwoods. Sparta. TN 

Cumberland Hardwoods was a nonunion rural plant. Because the company had a 
successful education program for several years, many of the employees had 
already passed through the available training courses prior to the study. 
Consequently, the samples at this site were quite small. For example, the GED 
course had so few students that insufficient data were obtained. Furthermore, no 
control groups could be formed. The population sample studied (N= 21) were 
employees taking part in a course, Communication and Collaboration, designed to 
help them work cooperatively as self-directed teams. Participants were members 
of two such teams. 
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Variables: Learner literacy variables included changes in beliefs (e.g., self 
esteem); changes in practices (e.g., reading and writing at home and work); 
changes in reported reading process and reading comprehension; and changes in 
plans (e.g., educational). Other measures were employer objectives such as 
employee attendance and ratings of supervisors. The researchers attempted to 
measure some family literacy variables, but the numbers were too small for 
statistical analysis. 

Design : Originally this study planned a “complete” experimental design, but small 
sample sizes precluded that possibility. As a result, some of the data were 
analyzed using statistical comparisons of quantitative information. Other 
quantitative, as well as some qualitative, data were analyzed by categories that 
emerged from interviews and open-ended questions. 

Data Collection: Interview, test, and classroom questionnaire data were collected 
before and after each course or at appropriate intervals during the class semester. 
Interviews were conducted by the researcher on a one-to-one basis. The classroom 
teachers administered the questionnaire and cloze test during the class period. 
Research personnel and on-site coordinators conducted class observations. 
Notations of class activities were documented on a timed basis. 

Instrumentation: The researchers used a combination of approaches to develop 
instruments, including literature reviews, surveys of indices of productivity at 
work, and discussions with plant supervisors to develop anchored rating scales. 
Susan Lytle’s conceptual framework for changes in adult literacy was used to 
develop interview and questionnaire items. These items were designed to reflect 
local use. Interview protocol forms, questionnaires, cloze tests, and classroom 
observations sheets were developed. Learners’ reading abilities and literacy 
practices at work and outside were measured by a questionnaire that used Likert 
scales, fill-in-the-blank items, and anchored rating scales. Cloze tests were used to 
gauge workplace reading comprehension. The family literacy interview protocol 
consisted of open-ended questions. An ESL competency checklist was also 
devised. Supervisor productivity rating forms were sometimes left with the 
supervisors and at other times completed with a researcher. 

Findings: Major results on the impact of employee participation in workplace 
literacy programs, excluding findings from the ESL class, are summarized below. 
Changes in variables were judged statistically significant when compared to the 
control group. 
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Changes in beliefs : The technical preparation group showed a statistically 
significant improvement from pre- to post-interview when asked, “How literate do 
you consider yourself to be?” Self-rating and holistic scores were statistically 
significant at the p < .02 and p <. 01 levels. The written questionnaire also 
contained a question about literacy beliefs. The latter data did not depict 
significant results, suggesting an advantage of the interview technique over the 
questionnaire method. Participants did not significantly change their views about 
what constitutes a literate person. 

Changes in practices : The Cumberland samples showed a significant gain in the 
task of “asking questions at work.” Reading and writing practices away from 
work were significant for the technical preparation group. There was also a 
significant gain in the range of reading for GED students. 

Except for the technical preparation group (because it was a full-time class and 
participants did not work while attending), all groups showed pre-post gains in 
work-related reading and writing. Participant responses were measured by a count 
of items measured and holistically-the frequency, breadth, and difficulty of 
reading. However, learner self-ratings showed very little gains. Nonetheless, they 
showed significant increases (p < .05) on two items, talking in meetings and 
having one’s ideas discussed in meetings. The Cumberland Communication and 
Collaboration class also showed gains in working cooperatively with others. 
Changes in literacy activities away from work were also measured. The technical 
preparation class showed statistically significant differences and the GED class 
showed gains. The GED class also showed a significantly wider range of reading. 

Changes in process and ability : Changes in reading process and reading 
comprehension were measured by interview questions and cloze tests constructed 
from plant reading materials. The Delco technical preparation class made 
statistically significant gains on the cloze test. Others did not, perhaps because the 
reading passages were too difficult. Significant gains were demonstrated on some 
of the interview items. 

When participants were asked to read a passage and describe what processes they 
used to read it, the posttest answers were deemed statistically significant for the 
Delco GED groups (p < .005). Their answers covered reading strategies (e.g., 
skimming) and topics of interest (e.g., products manufactured by competitors). 

Changes in plans : When asked about their plans for the future in one, five, and ten 
years and to relate reading to those plans, the technical preparation students 
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showed significant improvements for one- and five-year plans (p < .02 and p <. 
05). 

The other items measured were either not outcome variables or could not be 
accurately measured due to small numbers. Supervisor ratings at the Cumberland 
plant showed statistically significant gains. 

Strengths: Some population samples had control groups. The study generated 
quantitative and qualitative data. The design and measurement instruments were 
solid. The design included various modes to gather data. Comprehensive variables 
were measured in several ways. The study evaluated various ways to collect data 
from workplace literacy programs. 

Weaknesses: Some of the results were self-reported. Not all population samples 
had control groups. The sample sizes were small. Only data from two workplace 
literacy programs were included. It is questionable whether the Cumberland site 
should have been used at all, given that most employees had already passed 
through the program. It is difficult to understand how data gathered from just 
these two sites would effectively help develop and refine a research model for use 
at other sites. 

Mott Community College. (1993). Workplace Literacy Project. Computer aided 
instruction in basic workplace skills, May 1, 1991-October 31, 1992. 
Performance report. Executive summary. Flint, MI: Author. (ERIC Document 
Reproduction Service No. ED 378 308) 

Purpose: The Mott Community College (MCC) Workplace Literacy Project was 
designed to provide basic workplace skills to employees from Lapeer and Genesee 
Counties. Individualized training plans were developed for each employee. They 
were then placed in classes customized to the needs of the workplace and 
individualized to their assessed skills. The classes offered were basic 
mathematics, beginning algebra, intermediate algebra, college algebra, analytical 
geometry/calculus I, basic sentence skills, basic writing, English composition, 
reading improvement, and reading-study skills development. 

Variables: Variables measured were reading, writing, and math for work-related 
activities, critical-thinking and problem-solving skills in workplace situations, and 
interpersonal and communication skills in workplace interactions. 

Population: Five hundred and sixty-six employees from 20 companies were 
identified for participation. It was not clear how many of these employees actually 
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enrolled in classes or how many completed classes. Furthermore, many classes 
had duplicate counts. Thirty-nine percent were between the ages of 20 and 30; 
35% were between 21 and 40; 17% were between 41 and 50; and 7% were over 
51 years of age. Sixty-four percent were males. Only 6% had not received a GED 
or high school diploma. Eighty-six percent were White, 9% Black, and 3% were 
Hispanic or Native American. 

Design: The study had a pre-post design. Training was formatively evaluated at 
the midpoint of each training sequence and summatively evaluated at the end of 
each training sequence. The partnerships between MCC and the 20 work sites had 
a substantial variability of participation. For example, two partners had only one 
employee in the program, while another had 86. The average participation per 
partner was 28.3. 

Data Collection: Once employees were identified, they completed a battery of 
assessment tests. Supervisors were asked to fill out a five-point scale survey to 
rate employees. 

Instrumentation: Instruments used were MCC’s placement examination, the 
TABE, and the Myers-Briggs Personality Test. Only 214 participants were given 
placement tests. 

Findings: Narrative descriptions of outcome data were almost nonexistent, 
although statistical tables were presented. T tests for paired samples were 
provided. Performance in each academic area improved significantly, with math 
showing the greatest difference and vocabulary and comprehension showing the 
least. 

Supervisor ratings indicated that all partners considered the goal of helping 
employees advance in their careers at least moderately well fulfilled. Most also 
felt it increased productivity. Supervisors representing 70% of participants noted 
that 26.4% had extreme improvement in their self-esteem, 36% had extreme 
improvement in worker moral, and 27.8% had extreme improvement in attitude 
toward continuing their education. 

Strengths: There were no real strengths. 

Weaknesses: Very few of the participants had real outcome data. There was 
substantial duplicated counting of participants. The data generated were poorly 
described. Some of the data reported were subjective views of supervisors. It was 



NCSALL Reports #6A 



January 1999 



not clear what some of the placement instruments were measuring. There was 
substantial variability of employee participation in the 20 companies. 

National Center for Family Literacy. (1992) Dissemination process 
submission. Louisville, KY : Author. 

Purpose: This submission was a proposal for a U.S. DOE dissemination grant 
which included some evaluative data on family literacy programs using the 
Kenan model. Adult literacy education was a required component of all Kenan 
model programs. 

Variables: The National Center for Family Literacy (NCFL) had a comprehensive 
evaluation plan which included outcome measures for children as well as adults. 
The evaluation included tested learning gain, public assistance, employment, and 
further education. 

Population: The study population varied according to the program site. All 
subjects were engaged in adult literacy education. The Kenan Family Literacy 
Model Programs had seven sites with 122 students; sites in Hawaii had 53 
students; a site in Salem, Oregon had 1 8 students; and one in Mesa, Arizona had 
50. 

Design: Except for tested learning gain, most of the data were descriptive. There 
was a pre-post follow up, but the time between baseline and follow up varied by 
program. No control or comparison group was used. 

Data Collection: Data collection procedures were not explained in detail. The 
length of time between baseline and follow up was an average of 22 weeks of 
attendance at the Kenan model programs, two years at the Hawaiian sites, and one 
year at Salem, Oregon. The length of time was not reported for the Mesa, Arizona 
site. 

Instrumentation: An intake survey was used. Follow up and tested learning gain 
were measured by the TABE or CASAS. 

Findings: Kenan Family Literacy Model Programs : Grade-level gains were 
reported for reading, mathematics, and language. In reading, 41% gained less than 
one year, 30% gained one year, and 29% gained two to three years. In math, 20% 
gained less than one year, 25% gained one to two years, 24% gained two to three 
years, and 24% gained more than three years. In language, 20% gained less than 
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one year, 25% gained between one and two years, 10% gained two to three years, 
and 45% gained more than 3 years. 

Fourteen learners passed the GED; 41% were in some form of higher or 
continuing education; 66% were enrolled in educational programs, had definite 
plans for further education, or were employed; and 35% were employed (versus 
less than 10% at enrollment). In addition, 41% were not receiving public 
assistance, at enrollment 34% were not. 

Hawaiian sites : At the Hawaiian Competency-Based High School Program 
students could take either the GED or earn a school district diploma. Of the 29 
students enrolled in the first two years who did not have a diploma, all but four 
received high school certification. At one site, all adults ( n = 22) who completed 
the year enrolled in summer programs at the community college. 

Salem. Oregon : Seven of the 1 8 passed the GED. The average CASAS gain in 
reading was 13.5 scale points and the average CASAS gain in math was 16.5 scale 
points. Five students obtained a driver’s license, 13 registered to vote for the first 
time, four enrolled in regular programs at the community college, and four entered 
work for the first time. 

Mesa. Arizona : Of the 50 adults in the program, seven entered work for the first 
time, eight continued in community college ABE after their children entered 
kindergarten, six obtained their GED, and 20 remained in the program. 

Strengths: Comprehensive variables were measured. 

Weaknesses: The report did not mention details on methodology or context. Most 
data appeared to be self-reported. There was no control group. 

New York City Board of Education. (1985) Adult basic education final report 
1984-85. OEA evaluation report. Brooklyn, NY: Author. (ERIC Document 
Reproduction Service No. ED 278 793) 

Purpose: The Adult Basic Education/High School Equivalency (ABE/HSE) 
Services Program of the Board of Education’s Office of Adult and Continuing 
Education provided services to adults at over 300 sites. Its primary goal was to 
help program participants attain a higher level of literacy and enroll in other 
programs to further their education. This report compared variables measured in 
1983-84 and 1984-85. 
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Variables: Variables studied were reading and math gains for basic education 
(BE) students, and English language oral proficiency for ESOL students. Changes 
in course levels (e.g., from BE1 To BE2) and changes in student employment 
patterns were measured for both groups. 

Population: The sample of students consisted of those who enrolled in both the 
1983-84 and 1984-85 program years. The sample of 6,355 students was 33% of 
the total enrolled during 1983-84. Course-level advancement data were available 
for 1 ,302 BE students. Math scores were available for only 345 BE students and 
reading scores for 1,153. HSE program math scores were available for 109 
students and reading scores for 299. Employment status data were available for 
5,466 students. 

Design: Evaluators conducted secondary analyses on data compiled from program 
reports. A number of students and teachers were interviewed to gather qualitative 
data. 

Data Collection: Evaluators collected achievement test data from program files 
and reports. 

Instrumentation: The JOHN test was used to measure ESL oral proficiency. The 
TABE measured reading and math gains. 

Findings: Of the 406 BE students who entered at level one in 1983-84, 42% 
progressed one level, 4% two levels, and 1% three levels. Of the 439 BE students 
who entered at Level Two in 1983-84, 37% progressed one level, 43% two levels, 
and 1 6% three levels. The average grade equivalent score in math for BE students 
was 5.4 in 1983-84 and 5.6 in 1984-85. The difference in students’ grade- 
equivalent math scores was not educationally meaningful. The average grade- 
equivalent score in reading for BE students was 4.8 in 1983-84 and 6.0 in 1984- 
85. The comparison in students’ grade-equivalent math scores showed a moderate 
improvement in test scores. HSE program scores showed decline from one year to 
the next. In 1983-84 the average math score was 6.7, in 1984-85 it was 5.9. 
Average reading scores dropped from 8.3 to 7.7. 

Thirteen percent of the 2,5 1 8 students who were unemployed and available for 
work in 1983-84 were employed in 1984-85. Of the 1,157 students who had been 
unemployed and unavailable for work, 9% became employed by the 1984-85 
school year. 

Strengths: The sample was large and a huge amount of data was analyzed. 
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Weaknesses: The attrition rate was high. Very few variables were measured. Much 
of the data were incomplete. 

Orangeburg-Calhoun Technical College. (1995). VISIONS for Greater 
Employment Opportunities. Final report Orangeburg, SC: Author. (ERIC 
Document Reproduction Service No. ED 379 511) 

Purpose: The VISIONS project was a workplace literacy program conducted in 
partnership with Orangeburg-Calhoun Technical College. Participants were drawn 
from three companies. The project had five objectives, including the design and 
development of job-specific literacy audits and curriculum. Another objective was 
to provide instruction to 250 employees which would result in new or continued 
employment, career advancement, or increased productivity. This project also 
sought to increase the basic skills competencies of 90% of the participants. 

Variables: Outcome variables were competencies in speaking, listening, 
reasoning, and solving problems. 

Population: Two hundred and ninety-four adults participated in the workplace 
classes. More than 90% were between the ages of 26 and 65; 37% were White, 
61% Black, and 66% were female. Sixty-two percent of the participants were 
married and 85% had attained their high school diploma or GED before taking 
part in the classes. All were employed by the Regional Medical Center of 
Orangeburg and Calhoun Counties; United Technologies, an automobile assembly 
plant; or Holman, a cement and mining operation. The latter two were located in 
very rural areas. 

Design: The design was pre-post comparison. Upon completion of a task analysis, 
a competency- based curriculum was developed for each work site. 

Data Collection: Data were collected by project staff. 

Instrumentation: No standardized tests were used. All instruments that measured 
gains were competency-based and developed by project staff (primarily the people 
teaching the courses). All other documentation to collect data were created by 
project staff. The forms to measure participant outcomes included supervisor pre- 
post employee ratings and student interview or student evaluation of instruction. 

Findings: Ninety-nine percent of the participants attained a gain. There was a 57% 
average gain in better job attitude and a 58% gain in job knowledge. As reported 
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by supervisors, participants showed a 50% gain in productivity. Self-assessment 
surveys indicated that instruction helped workers perform their jobs better, with 
ratings increasing from 72% in the first cycle to 91% in the fourth cycle. 

Strengths: Useful materials and courses were developed for the three participating 
companies. 

Weaknesses: Some participants were double counted. The lack of standardized 
testing makes it difficult to generalize this approach and its results. Teachers 
administered the pre- and posttests that they had developed. There was no control 
group. 

Pacific Training and Technical Assistance Corp. (1974). Evaluation of the 
community based Right to Read program. Berkeley CA: Author. 

Purpose: This study was a U.S. DOE-sponsored evaluation of the Right to Read 
program. The evaluation was quite comprehensive, looking at program effects as 
well as learner outcomes. The evaluation focused on 13 programs serving in- 
school youth and 1 1 programs serving out-of- school adults. Data on the adult 
projects are reported here. 

Variables: The variables studied were tested learning gains, students 
characteristics, and program characteristics. 

Population: The study population consisted of Right to Read participants in 1 1 
randomly selected sites. The cumulative enrollment was 1,198 students. Six 
hundred and forty participants were continuees and 322 received pre- and 
posttests (a completion rate of .50). 

Design: The design was pre-post comparison. Participants were administered 
reading tests and gains were measured against student, teacher, and program 
characteristics. 

Data Collection: Pretests were administered in November and December 1973. 
Project staff monitored the testing. Posttests were administered in May 1974. 
Student and staff data were collected bimonthly until May 1974. 

Instrumentation: The Reading Everyday Activities in Life (REAL) test was used 
to measure learning gain. Learners’ attitudes toward reading were measured with 
an attitude scale developed by the researchers. Student data were collected 
through intake and termination forms. Other instruments were attendance data 
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forms, a staff questionnaire, an on-site observation guide, and a project director’s 
interview guide. 

Findings: Significant gains in reading occurred. Average gains were 6.15 in the 
four- to six- month period. There were no significant differences between 
programs using paid tutors and those using volunteer tutors. The termination rate 
was 30.9%. Those who could read in their native language gained more than those 
who could not. Nonnative speakers who were fluent in English did better than 
those who were not fluent. Those who perceived their participation to be 
volunteer did better than these who did not. 

Strengths: The sample size was substantial. The design was well conceived and 
executed. Comprehensive variables were measured. It was a well-documented 
report. 

Weaknesses: There was no control group. 

Payne, B. (1989). Project F.I.S.T.: Functional In-Service Training. Submission 
for program recertification. Edison, NJ: Middlesex County College. 

Purpose: This was the application for Joint Dissemination Review Panel 
recertification for Project F.I.S.T., a volunteer-based administrative and 
instructional delivery system aimed at meeting the needs of low level readers. 
Student and tutor outcomes were measured. Solely the latter is discussed in this 
abstract. 

Variables: Student outcomes measured were reading achievement and change in 
reading behaviors. 

Population: Fourteen (14) were from the home site and four from an adoption site 
in Maine (n=14). The replication group (second data set) consisted of five 
students from the home site and 48 from the Literacy Assistance Center (LAC) 
database. 

Design: A single group pre-post design with replication was employed. 

Data Collection: Data were collected by project staff. The standardized reading 
test was administered at entry level and at four-month intervals. 

Instrumentation: The SORT and TABE were the standardized tests used. A 
questionnaire was also administered. 
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Findings: Students made significant progress on the SORT. They averaged eight 
months of improvement in four months. In addition, when compared to the data 
from LAC (of 1 10 students and 3.0-4.9 grade level), the average gain for F.I.S.T. 
students on the TABE was 1 . 1 after four months of instruction versus the same 
gain after 200 hours for the LAC sample. 

Strengths: Results were compared with those of a nationally validated program 
and the LAC database. 

Weaknesses: There was a low n. The sample of students was described in a 
confusing manner. Three different standardized reading tests were used. 

Ramussen, C. (1992). Workplace basic skills in the metal casting industry for 
world class process and technology. Alexander City, AL: Central Alabama 
Community College. 

Purpose: Between March 1991 and August 1992, Central Alabama Community 
College and Robinson Foundry, Inc. established a workplace basic skills project 
for the metal casting industry. This project sought to measure seven employee 
objectives and four industry objectives. The two institutions wanted to establish a 
correlation between literacy gains and productivity measures. 

Variables: Employee outcome variables were literacy achievement, knowledge 
and performance of industrial process, and GED completion. 

Population: Three hundred and three workers enrolled; 216 completed all 16 
hours of instruction and had pre-post data. 

Design: The design was descriptive with a pre-post component. 

Data Collection: Exit surveys were administered during the second and third 
reporting periods. Data collection methods were not described. 

Instrumentation: The TABE was used to measure general literacy. Instructors 
developed pre- and posttests to measure knowledge of industrial process and 
concepts of management. Performance of the industrial process was measured by 
surveys that the instructors generated based on a supervisor rating form developed 
by a workplace literacy consultant. 

Findings: A correlation between literacy gains and productivity measures was not 
established. No one attained a GED. The TABE scores were difficult to interpret; 
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however, reported reading gains were two to three grade levels. Seventy to 85% 
of participants perceived the project to have helped them with reading, writing, 
computation, and problem solving; and 70% said it had helped job performance. 
Job attitude improved by 71% and job knowledge by 62%. Retention was 71%. 

Strengths: There were no strengths. 

Weaknesses: The treatment period, 16 hours, was short. The researchers did not 
provide samples or descriptions of instructor-developed tests. Much of the data 
were self-reported. 

Scheier, E., & Senter, D. (1969). Evaluation of Learning 100: An adult basic 
education project in Bedford-Stuyvesant, 1967-68. Huntington, NY: 
Educational Development Laboratories, Inc. 

Purpose: L-100 was a multimedia communications skills system developed by 
Educational Development Laboratories for use with undereducated adults and 
youth. This study sought to compare its effectiveness in an undereducated adult 
population from the inner city with a more traditional reading program. 

Variables: Reading achievement and hours of instruction attended were measured. 

Population: Forty-nine students enrolled in the two L-100 classes. A control 
group of 47 enrolled in two classes that used traditional reading curricula, such as 
SRA Kits and the Lippincott Reading for Meaning Series. Only 37 of the 
experimental group and 26 of the control group had pre-post data. Sixty percent of 
the students in the experimental group had completed 4-7 years of schooling, 
while 62% of the control group had completed 9- 1 2 years of schooling. All the 
members of the experimental group were employed, as were 73% of the control 
group. Most of the students from both groups were women. Sixty-two percent of 
the experimental group were between 35 and 49 years of age; 65% of the control 
group were between 20 and 34 years of age. 

Design: This study had an experimental design. The analysis of covariance 
statistical technique was used. Qualitative data were classified and summarized. 

Data Collection: The Metropolitan Achievement Test (MAT) was administered 
after 1 0 hours of instruction, after 1 00 hours of instruction, and at 200 hours of 
instruction. Research staff administered questionnaires to teachers and 
interviewed the teachers. 
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Instrumentation: The MAT was used to measure reading achievement. Subjective 
data were collected by questionnaires completed by and interviews given to the 
teaching staff. 

Findings: Twelve (24%) of the experimental group and 21 (44%) of the control 
group left the program without completing the prescribed 200 hours. No 
correlation was made between retention rates and method of instruction. The 
average hours of instruction for the experimental group was 157 and 170 hours for 
the control group. Again, no correlation was made between average hours of 
instruction and method of instruction. The experimental group scored significantly 
higher on the MAT than did the control group. 

Strengths: There was a control group. There was a more than adequate treatment 
period. Statistical adjustments were made for pretest differences. Variables were 
correlated statistically. 

Weaknesses: There were small sample sizes in each group. The experimental and 
control groups did not seem comparable in several areas. Few variables were 
measured. The reading instrument was not normed for adults. 

Seaman, D., & Denyer, J. (1991). Evaluation of a workplace literacy program: 
A cooperative effort between Houston Lighting and Power Company and North 
Harris County College. College Station, TX: Texas A&M University. 

Purpose: Between 1990 and 1991, Houston Lighting and Power Company, in 
conjunction with North Harris County College, developed a workplace literacy 
project to prepare the company’s employees for the commercial driver’s license 
(CDL) test. 

Variables: Variables were reading achievement, passing the CDL test, and desire 
to further one’s learning. 

Population: Originally the project meant to serve 200 employees in the pilot and 
regular classes. This goal was not met due to lack of release time by the employer. 
The report did not make clear how many employees actually participated. The 
target pool was originally 1,200 employees, of which 1,190 were assessed by one 
of two instruments and 500 participated in a one-day class on “test-taking skills.” 

Design: The report was primarily descriptive with a pre-post component. 

Data Collection: It was not clear who collected data or how it was collected. 
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Instrumentation: Instruments used were the CDL Basic Skills Assessment and the 
TABE. 

Findings: Of those who enrolled in the regular class, 99% passed the CDL test. 

Strengths: Independent evaluators assessed the project’s pilot class and the main 
instructional program that followed. Evaluation was based on Steele’s (1990) 
four-component model. 

Weaknesses: There was no control or comparison group. The sample size was not 
clear. Data collection methods were not clearly described. 

Sherron, R. (1986). A longitudinal evaluation of adult basic education in 
Virginia: 1970-1985. Richmond VA: Virginia State Department of Education. 

Purpose: This research was actually a cohort study, although it was called a 
longitudinal evaluation. Similar instrumentation was used in 1970, 1978, and 
1985, and the results were compared. Most of the data described program and 
teacher variables. Some data on student impact were presented. 

Variables: For the three cohort years, these variables were compared: income, 
employment, reasons for nonemployment, residence moves, post-ABE training, 
attendance, and learning gain. 

Population: There was a 10% random sample of students. The actual sample size 
was not given. 

Design: The design was a descriptive cohort study with cohorts in 1970, 1978, 
and 1985. 

Data Collection: The methods and procedures of survey administration were not 
described. 

Instrumentation: The test used for tested leaning gain was not described. Other 
data were collected through survey. 

Findings: Students’ income increased over time as did students’ family income. It 
was not reported whether the data were adjusted for inflation. Full-time 
employment decreased over time. Reasons for nonemployment remained stable 
over time. Over time, more students did not change their residence within a five- 
year period. More students expected to attend a college or university over time; 
preferences for other types of post-ABE training did not change. In 1970, 
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significant gains in both reading and math were achieved; in 1978, gains in 
reading were not significant, but gains in math were. In 1985, gains in reading 
were not significant, but gains in math were. Neither the test used nor the time 
interval between pre- and posttesting were described. 

Strengths: There were time series data and comprehensive variables. 

Weaknesses: Methods and procedures were not adequately explained. This was a 
cohort study, not a longitudinal study. Without a control group, gains over time 
can not be explained by participation in ABE. 

Siefer, N., & Latkiewicz, J. (1992). Third project evaluation report-summative 
for Salt Lake Community College. Salt Lake City, UT: Salt Lake Community 
College. (ERIC Document Reproduction Service No. ED 355 373) 

Purpose: Salt Lake Community College, the National Semiconductor 
Corporation, and Natter Manufacturing, Inc. collaborated in implementing a 
workplace literacy project. This project had eight objectives, including conducting 
a literacy audit, performing task analysis for selected jobs, designing and 
implementing curriculum, and evaluating and disseminating the results. It was 
originally designed to serve 225 employees. 

Variables: Variables measured were impact on employees’ workplace literacy 
levels, workplace technological skills, workplace communication skills, 
participation in team meetings, and job advancement. 

Population: Three hundred and twenty-five employees from the National 
Semiconductor Corporation enrolled in Pre-Statistical Process Control (SPC), 

ESL, math, and technical reading and writing classes. Thirty-five employees from 
Natter Manufacturing attended classes in math, communication, and ESL. Most of 
them enrolled in the ESL class and many of them enrolled in more than one class. 

Design: There was a pre-post design. The evaluation plan for this project 
generated soft and hard data. 

Data Collection: Project staff collected the data. Procedures were not made clear. 

Instrumentation: During the project it was difficult to pretest for the course 
objectives and project personnel did not want to rely on generic skill tests. 
Checklists, supervisor ratings (four- item scale), personal interviews, and 
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anecdotal summaries were used to collect data and assess the impact of the 
project. 

Findings: Natter Manufacturing : Of the 35 enrolled in refresher math, 26% 
showed gain to the 80% mastery level. Of the 19 enrolled in the workplace 
communication class, 37% showed gain to the 80% mastery level. Five people 
(71% of the class) showed gain to the 80% mastery level in technical reading and 
writing. 

National Semiconductor : Of the 137 who attended the refresher math course, 82% 
showed some gain, and 63% achieved mastery level. Of the 101 participants who 
took both pre- and posttests, 86% showed gain and 53% achieved mastery level. 

Self-reported data indicated that some employees were reading the newspaper 
more frequently, were more involved with their children’s homework, and had 
increased self-esteem. 

Strengths: This evaluation study had no real strengths. 

Weaknesses: Many classes met for so few hours that true impact could not be 
measured or attributed to participation in the classes. Learner count was 
duplicated in many cases. 

Smith, M., & Dalheim, Z. (1990). Project READ. A study of twenty reading 
disabled adults. Pittsfield, MA: Pittsfield Public Schools. (ERIC Document 
Reproduction Service No. ED 328 726) 

Purpose: Project READ’S teaching objective was to improve the reading levels of 
adults diagnosed with learning disabilities. Its three research objectives were to (a) 
investigate which of three teaching approaches was most effective with this 
population, (b) study what is effective assessment, and (c) measure the impact of 
reading difficulty and subsequent remediation on an adult’s self-esteem. 

Variables: The outcome variables were reading gains, the impact of instruction on 
self-esteem, and changes in students’ lives. 

Population: Initially 25 students were selected for the project. Their ages ranged 
from 20 to 67 years. Eighty percent were men. Most were employed. Seventeen 
had school-aged children. All scored below fifth grade on a battery of tests. All 
had intelligence in the normal range and English was everyone’s first language. 
Five dropped out before completing the program ( N = 20). 
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Design: The project had pre- and posttesting over time and a case-study design. 

Data Collection: Project READ took place and data were collected from 1986 to 
1990. Data were collected at the beginning of the project and at its conclusion. 
However, it is not clear at exactly what other intervals collection occurred. 

Instrumentation: The study used a variety of instruments. Standardized tests 
included the Wechsler Adult Intelligence Scale to measure intelligence and 
discrepancy between reading accuracy and oral comprehension. Subtests from the 
WAIS were used as auditory and visual memory measurements. The Wide Range 
Achievement Test was used to measure word recognition, spelling and writing, 
and the Woodcock Reading Battery to assess word attack skills. Silent reading 
was measured by the Stanford Diagnostic Reading Test or the TABE; oral reading 
was assessed by Gary Oral Paragraphs and Gilmore Oral Paragraphs. Self-esteem 
and other psychological variables were measured by the Depression Adjective 
Checklist, the Gordon Personal Profile Inventory, and the IP AT Anxiety Test. 
Project staff created intake and completion interview instruments. In-depth case 
studies were kept on many of the students. 

Findings: Although the samples were small, this study produced many results 
because so many types of comparisons were made and reading strategies assessed. 
Consequently, there were too many outcomes to delineate in this abstract. The 
major findings are noted below. 

Reading gains were measured by grade-level gain and the number of lessons 
required to attain it. The average grade-level gain for the CC group was one year 
and three months after 210 lessons; the RFS grade-level gain was three years and 
three months after 203 lessons. Compared to the average gain in public school, the 
CC group was 1.5 times the average and the RFS group 3.1 times the average. 

Oral and silent comprehension were a key area of difference between the CC and 
RFS groups. The CC group gained eight months while the RFS groups had a gain 
of four years. This was a rate gain of 4.6 times the average. 

The RFS and EQ groups were compared for reading gains on untimed oral tests. 
The EQ group gained three years and six months after 184 lessons. The RFS 
groups gained three years and nine months after 216 lessons. When compared to 
public school averages, the EQ group was 4.0 times the average and the RFS 
group was 3.3 times the average. Again, the greatest differences were found in 
comprehension. The EQ groups had a rate gain of 5.1 versus 3.6 for RFS. In silent 
reading the two groups were almost identical with nearly a 4.0 gain for both. 
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The groups showed greater gains in comprehension than in phonetic and word 
structure decoding, even though the curriculum stressed these latter skills. The 
researchers concluded that students with learning disabilities obtain sufficient 
phonological understanding and have enough practice in the use of phonological 
and word structure clues that it frees them to attend to comprehension. This 
conclusion ran counter to the accepted beliefs about phonics instruction. 

More than 50% of the students indicated that their plans for the future were 
different than when they enrolled. Seventy percent of participants noted that they 
had higher self-esteem and 60% had job changes which were upwardly mobile. 

Strengths: This study had an adequate design. Three rigorously selected samples 
were compared to one another. Many variables were analyzed, compared, and 
correlated. The data collected generated quantitative and qualitative information. 
This type of study can help literacy practitioners in the classroom. 

Weaknesses: There were too many assessment instruments. There were low 
sample sizes. Description of data collection methods was weak. Too many reading 
skill variables were analyzed and some seemed redundant. Because of the way it 
was structured, Project READ and this study would be difficult to replicate. 

Snoddy, J. (1990). Macomb Reading Partners research report Clemens, MI: 
Macomb County Library. (ERIC Document Reproduction Service No. ED 
380 560) 

Purpose: This study evaluated Macomb Reading Partners (MRP), a one-to-one 
volunteer tutoring program of the Macomb Literacy Project. The primary 
objective of the study was to examine the reported motivations of adults with low 
levels of literacy who seek entry into a literacy program. It also sought to 
investigate the conclusions of Aslanian and Brickell (1980), who found that adults 
in transition or experiencing a trigger event (such as divorce, being laid off, 
finding a new job, or illness) tend to seek new learning experiences. The two 
major research questions were: (a) What are the motivations reported by low- 
literate adults for entering a volunteer literacy program; and (b) What are the 
levels of literacy of the adults in this study? The study also focused on the 
employment patterns of participants and their coping or concealing strategies in a 
literate world. 

Variables: The outcome variables were gains in reading grade levels and the 
application of literacy skills in daily lives. The study also summarized participant 
responses to the questions: “Do you feel better about yourself?” and “Why or in 
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what ways do you feel better about yourself?” The responses were inductively 
coded and grouped into the categories of improved literacy, enhanced self- 
confidence or self-esteem, personal accomplishment, and enhanced job 
opportunities or performance. 

Population: Selected participants had to have been enrolled in the literacy 
program for at least six months and have no mental, emotional, auditory, visual, 
or neurological impairments. Students’ grade-level reading equivalent ranged 
from primer to fifth grade. Seventy-three percent were men and the mean age was 
38. Seventeen percent had completed grades 6-8, 40% grades 9-11, and 43% 
completed high school. Most were employed and approximately 60% worked in 
the automotive field. 

The sample was selected from the list of participants who remained after the first 
two criteria were met. Subjects were then selected randomly and contacted by 
telephone about their availability and willingness to participate. This procedure 
continued until a sample of 30 was selected. The researchers believed this sample 
was representative of all participants in the program. 

Design: The design was pre-post comparison. Quantitative data were reported and 
analyzed by using frequency counts, measures of central tendency, and other 
descriptive comparisons. The data generated from the selected sample were 
compared with the general population of MRP. 

Data Collection: This study lasted approximately one year. Planning began in 
October 1989, data collection ended in July 1990, and data analysis was 
completed by September 1990. Program participant files were used to collect 
information about experiences with tutoring and to validate information gained 
during interviews. Interviews were conducted by the researchers at the beginning 
of the study and at its completion. Interviews generally lasted an hour. 

Researchers took extensive notes throughout the interviews (they were not taped). 
Responses were inductively coded and grouped into categories. A comparison 
group was not interviewed. 

All students at MRP were tested at six-month intervals, a procedure that continued 
throughout the study. Initially students were tested by a trained member of the 
staff. Thereafter, the tutor administered the test. Program staff scored and 
recorded the results. 

Instrumentation: The Word Opposites Test of the Botel Reading Inventory was 
used to measure reading gains. Life skills and self-esteem were explored by an 
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interview protocol devised by the researchers based on a study by Darkenwald and 
Valentine (1984). Four pilot interviews were conducted and the protocol was 
revised slightly. 

Findings: The average reading gain per month was .204. In general, the results 
showed that the lower the grade-level equivalency of an adult at entry the greater 
the gain per month. Furthermore, the older student, aged 40-58, had higher per 
month achievements than younger participants. The average monthly gain was 
significantly higher for those tutored for 10 months or less than for those tutored 
13 months or more. Qualitative data indicated that many of the students had 
changed to more effective reading strategies. 

All participants answered affirmatively when asked, “Since you started this 
program, have you used your reading skills to do something you couldn’t do 
before, or to do it better?” 

Study subjects were asked to name some of those things. All the answers were 
delineated. The most common were reading a menu, mail, Bible, newspaper, and 
magazine article. Voting was also mentioned fairly often. Ninety-seven percent of 
the respondents reported feeling better about themselves. They also mentioned 
most often that feeling better was due to improved literacy and enhanced self- 
confidence or self-esteem. 

Strengths: In general, the research design was adequate. Quantitative data were 
measured against a comparison group. There was a solid interview protocol and 
rigorous analyses of qualitative data. Researchers based their study on and 
compared their results to previous research. 

Weaknesses: The sample was very small. Given the test used, true reading-gain 
results could be questionable (e.g., no reading in context). Some of the data were 
self-reported. 

Turner, R., & Stockdill, S. (Eds.). (1987). The Technology for Literacy Project 
evaluation. A special report from the Donor Review Board. St. Paul, MN: 

Saint Paul Foundation. (ERIC Document Reproduction Service No. ED 295 
028) 

Purpose: The Technology for Literacy Center (TLC) implemented a project 
designed to investigate and execute new, innovative ways to teach literacy skills 
to adults living in the greater St. Paul area. It had several components: direct 
service, staff development, research, and incentives grants to existing literacy 
organizations. TLC developed a competency-based program built on a mastery 
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system of instruction. The project implemented formative and summative 
evaluation plans. This abstract describes the student impact results of the 
summative evaluation. 

Variables: Impact variables were reading and mathematics achievement. Attrition 
rates and reactions to the TLC approach were also measured. 

Population: This study focused on 165 learners who enrolled between September 
and November of 1986. They were considered the summative group. Most of 
them were English-speaking adults who lived in the St. Paul tri-county area. 
Eight-one percent (81%) of the learners read below the eighth grade level at intake 
and 41% of them had not previously enrolled in a literacy program. Fifty-one 
percent were White, 28% Black, 8% Hispanic, 5% Asian, and 5% Native 
American. All were voluntary students. Only 50 were still in attendance when 
achievement gains were measured. 

Design: Formative and summative evaluation plans were implemented. The 
outcomes of the summative group were compared to a control group of other TLC 
enrollees. They were also compared to learners in local and nationally recognized 
programs. All of the achievement data gathered were reported quantitatively. 
Focus group data were analyzed qualitatively. 

Data Collection: A formative evaluator, employed by the supporting foundation, 
designed a system for data collection, and collected and analyzed most of the data. 
Data were regularly collected from student files, which were updated with each 
student visit. Data were aggregated quarterly. Students were given the TABE at 
enrollment and in April for everyone who had attended at least 1 2 hours of 
instruction. 

Focus groups were used to collect qualitative data. Three were conducted by the 
primary summative evaluator during TLC’s regular hours. The interviews 
followed a six-question protocol. A total of 20 people took part in the focus 
groups. 

Instrumentation: Instruments used to collect data were standardized achievement 
tests, group interviews, exit interviews, student questionnaires, learner activity 
logs, and case studies. The TABE was the standardized achievement test. 
Criterion-referenced tests were also developed to document progress on the 
computer-based modules. 
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Findings: The outcomes of the summative group were as follows (numbers were 
reported as means): 

The composite reading gain was .75 grade level for students who tested between 0 
and 4 on the pretest (n = 1 5). The gain in comprehension was .98 grade level. The 
vocabulary gain was .6 grade level. This group averaged 47.2 hours of instruction. 
It was calculated that 63.2 hours were needed to make a full grade-level change in 
reading, 48.2 hours were needed for a full grade- level change in comprehension, 
and 78.7 hours were needed for such a change in vocabulary. 

The composite reading gain was .85 grade level for students who tested between 5 
and 12 on the pretest (n - 23). The gain in comprehension was 1.13 grade level. 
The vocabulary gain was .79 grade level. This group averaged 44.5 hours of 
instruction. It was calculated that 52.5 hours were needed to make a full grade- 
level change in reading, 39.6 hours were needed for a full grade- level change in 
comprehension, and 56.6 hours were needed for a full grade-level change in 
vocabulary. In sum, the impact of TLC upon the summative group was positive 
for reading comprehension and math, but not for reading vocabulary. The control 
group showed no such gain in comprehension. 

The researchers also provided a comprehensive analysis of the impact of hours of 
instruction in relation to achievement gains. TLC’s results were also compared to 
the three validated NDN literacy programs. The researchers concluded that TLC’s 
reading gain results were “in the ballpark” with the three programs. However, the 
same conclusion was not reached for math gains. JCARP’s students showed 13.7 
hours per grade-level change on the ABLE compared to TLC’s 45.4 hours. 

Qualitative data indicated that TLC participants felt they were learning and many 
of the learners described the differences it had made in their lives— reading to their 
children, keeping pace with their children, enhanced self-esteem, and ability to be 
self-motivated. 

Strengths: This was a solid experimental study with a strong summative 
evaluation. Analysis of data was very comprehensive and well described. There 
was a control group. 

Weaknesses: The sample size was small for measuring impact gains. Hours of 
instruction were limited for many of the participants. Attrition rates were high. 
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Wasson, J. (1965). Evaluation of an eight week adult education program. St. 
Paul, MN: Ramsey County Welfare Department, Work and Training 
Program. (ERIC Document Reproduction Service No. ED 021 194) 

Purpose: The Ramsey County Welfare Department and the St. Paul Public 
Schools conducted an eight-week summer adult education program for public 
assistance recipients. The clients were referred to a work experience program, a 
work training program, or an education program. The purpose of the study was to 
describe and analyze the outcomes of the educational program, and relate those 
outcomes to student demographic and psychometric characteristics. 

Variables: The variables measured were reading and math gains, changes in work 
values, and student perception to the program. Changes in counselor description 
of clients and counselor perception of program were also measured. 

Population: The population of 34 males and 19 females were clients of the 
Ramsey County Welfare Department’s Work and Experience Project. All had an 
average IQ. All the males and one female were married. Three females were 
single and 15 divorced. The males ranged in age from 19 to 52 years, with a mean 
of 32.0. The females ranged from 23 to 46 years. Forty two of the participants 
were White and 1 0 were Black. Forty-one percent of the males and 79% of the 
females had been unemployed for more than 52 weeks. Highest grades completed 
were the following: 



Highest Grade Completed 


Males 


Females 


6th grade or less 


14.7% 




7th grade or less 


5.9% 




8th grade or less 


5.9% 


10.5% 


9th grade or less 


26.5% 


31.6% 


1 0th grade or less 


29.4% 


21.0% 


1 1th grade or less 


17.6% 


21.0% 


1 2th grade or less 


2.9% 


15.8% 



Design: The study had a pre-post comparison design. 

Data Collection: Reading, math, and psychometric tests were administered by a 
trained psychometrist in groups of 8 to 12 during two half-days of testing. 
Reading and math instruments were administered three times during the program 
period: at intake, during the first week of class, and during the last week of class. 
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Instrumentation: The Gates Reading Survey was administered to measure reading 
speed, vocabulary, and comprehension. The Jastak Wide Range Achievement Test 
measured computational ability. Both instruments yielded raw and grade scores. A 
scale to measure work- value changes was administered. It was a 53-item measure 
of work attitude developed by Dr. Arthur Bradely and the Minneapolis Veterans’ 
Administration Hospital. The Army General Classification Test measured 
intellectual capacity. Percentiles were transformed to IQ scores. The Minnesota 
Paper Formboard was used to test spatial reasoning. The Kruder Preference 
Record, Vocational, Form CH was administered to measure vocational preference. 

Findings: Reading speed increased from 7.9 to 1 1 .2 for the thirty participants who 
had valid pre- and posttest reading scores. Reading vocabulary increased from 9.3 
to 10.3 and reading comprehension from 9.5 to 10.5. The average grade-level 
increase of math scores was 1.92. Seventeen people took the GED test. Six passed 
and 1 1 failed. The data suggested that those who scored lower than 9.0 at intake 
were not able to pass the GED after eight weeks of instruction. All who read 10.0 
and better at intake passed the GED test. 

Upon completion of the course, 36% were either going directly to work or into an 
employment training course. These training programs included janitorial, dietary 
assistant, nurse’s aide, laundry, and orderly courses. Thirty-eight percent chose to 
enter educational programs. 

There were no significant changes in the students’ responses to the Kruder work- 
value scale. 

Strengths: Valid assessment instruments were used. Variables were correlated. 
The research design and data collection methods were adequate. 

Weaknesses: There was a small sample size. There was no control group. 

White, G. (1990). A comparative study of adult education 
Indianapolis/Richmond. Third party evaluation final report. Indianapolis, IN: 
Indiana Opportunities Industrialization Center of America State Council. 

Purpose: This study sought to compare the effectiveness of traditional classroom 
instruction to CAI for adults enrolled in ABE classes. Its purpose was to measure 
the effectiveness of the methods in raising the levels of basic skills after 80 hours 
of instruction. Effectiveness of instruction was also correlated with the various 
learner styles of the students. The CAI site was located in Indianapolis, IN; the 
traditional site in Richmond, VA. 
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Variables: Outcome variables measured and compared were reading and math 
achievement, retention rates, and self-esteem. 

Population: One hundred and forty-nine students enrolled, but only 50 met the 
criteria for comparison purposes. The majority were women ranging in age from 
1 8 to 67 years. The mean entry reading level on the TABE was the equivalent of 
7.5. 

Design: The design was pre-post comparison. 

Data Collection: Methods included classroom observation, student survey and 
interviews, reviews of instructional records and materials, joint site meetings, and 
interviews with project administrators. Data was collected upon entry, at 30-day 
intervals, and upon exit over a period of 1 8 months. 

Instrumentation: The ABLE was used. 

Findings: The average gain for CAI students was 2.6 grades and 1.84 for 
traditional students. 

Strengths: Effects of methods of instruction were compared. Effectiveness of 
methods of instruction was correlated with various student learning styles. 
Anecdotal information regarding workplace literacy programs was 
comprehensive. 

Weaknesses: Most data were self-reported. There was no clear research design. 
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